Will Open Source AI Overtake Closed Models? Ft. Olama, Fireworks and Open Router

So now we're welcoming Sonia Huang for our next panel on open source with Dima from Fireworks.

We got Alex Atallah from OpenRouter, and of course, Captain Jeffrey Morgan from Ollama to talk about the other side of the equation, the open source future that you all can use and plug into your products and adopt today. Please welcome them to the stage.

Thank you. Alex, Dima, Jeff, thank you so much for joining us. We heard Jensen speak this morning with an impassioned plea for the importance of a thriving open source model ecosystem. And I think everyone in this room is fully aligned behind that vision. So I'm delighted that we have this panel today. Let's do a first quick, first round of intros. Maybe say your name and then what your company does in a sentence or two.

Hi, I'm Jeff. Thanks for having me. I'm a founder of Ollama. It's the easiest way to get up and running with open source LLMs. And that's me.

Yeah. Hi, I'm Dima. I'm co-founder of Fireworks. And you heard from my co-founder, Lin, this morning also in the presentation. And yeah, Fireworks, as you heard in the morning, is the best place to build and scale this generative AI. We help you to fine-tune and serve models at really fast and large scale. And obviously we are very big proponents of open source models because that's what we're doing this. And yeah, I'm passing to Alex. We are happy to be one of the providers on OpenRouter too.

I'm Alex Atallah, CEO and co-founder of OpenRouter. We're a language model router and marketplace, a way to just have one service that gives you the best pricing, performance, and uptime for models. And we work closely with a lot of different providers, but in a very big way with Fireworks to make that happen.

Great. Okay. Let's start with the problem statement that Jensen framed for us this morning. Some of the fearmongers in the ecosystem are pushing to have open source models heavily regulated or banned. Obviously, the people in this panel probably don't agree with that. You guys are the champions of the open source ecosystem. Like, what would you say? How would you make the case for the importance of open source?

Go ahead. I think a fundamental principle behind open source is that human genius can still come from anywhere around the world, and that centralizing it behind model labs is very risky. And it's not as good of a bet. It's not as high leverage of a bet. As allowing genius to come from anywhere. And what I think like language models and generative AI unlocks is for the ability for somebody with access to unique intelligence that can't be found anywhere else to create a service that generates it on demand when anyone wants. And we're just gonna, I think we're in a period right now where open source is not winning. Compared to closed source, it's about 20% to 30% of our inference in terms of tokens used. But— and it's also like a last mile optimization for a lot of companies. Or like you have one customer that just like refuses to work with some closed source model provider, and you want to serve them anyway. But I think like the fact that it's transparent and the fact that it's very, very high leverage compared to closed source models, means that when the young kids of today grow up really thinking about state-of-the-art machine learning on the edge, they're just going to make something open source before they make something closed source because they won't be able to work at a model lab. And then we'll start seeing really wild things develop.

If I can add, I mean, it's a question of practicality. So on two fronts, one is a lot of the open source models are deployed on non-data center hardware. And so it ends up the customer has access to it anyways. For a consumer, there's a question of, well, if you're gonna distribute the model on their device, how hard is it for them to realistically access it? Kind of like, you know, maybe Photoshop in the old days or how operating systems are increasingly free for consumers. And often these models aren't fully open in the sense of the data and the training and the paper, but the weights are there. And so if you're downloading the model anyways, you're kind of getting the weights. On the enterprise front, a lot of these enterprises are really excited about customizing the model. And if, you know, these things are banned, but enterprises still want this, there's probably a way that that could still work. But then you'd be asking, well, the customer wouldn't, you know, really own the entire result. And from what we've learned, that's really important for customers, that when they fine-tune or train or distill a model with their own data, that they own the entire model, not some, you know, I don't know, piece of it. And so those are two kind of practical questions that I think we'd have to work through if that is the case.

To add to that, basically if you view AI as general purpose technology, even from global benefit in the society, you can try to regulate electricity or something. You can try to regulate applications and distribution, how the market works, but you shouldn't ban the basic technology. And I feel from that perspective, it's just for developing of scientific progress, it's useful. And to the point of customization, we also see a lot of people people wanting to customize and tune and actually build AI-first products. And that's where the same argument for open-source software, if it's a core part of your product, you want to customize it, you want to control it, you want to be basically with it.

Actually, on that note, show of hands, who here uses open models in their production stacks today? Okay, good amount of the crowd. Alex Atallah, I want to pull on the thread that you mentioned earlier, which is that the open-source models are, or the open models are behind right now. Say more about that dynamic and how do you think that'll play out?

So right now, I mean, I see that most of the large model labs trying to build more and more general powerful foundation models that you need to fine-tune less and that work for more use cases. I kind of, if I were to make a prediction now, I think that fine-tunes are maybe gonna have a weak moment in the next year or two. Year or so as these powerful RL models scale up and find like the marginal benefit you get from fine-tuning gets outweighed by the obsolescence risk from doing so. But we're just still at the very beginning, as everybody today has said, of a massive wave of adoption where just most businesses still haven't even tried anything. And when that wave is further along, I expect a very similar thing to happen that happened with software where people just want transparency and they want customizability. And then we might see like, I don't know if it'll be called fine-tuning in that future, but we may see some like, it has to be open-source type customization arrive that the open-source community will land on probably first.

And who do you think will be the stalwart of that kind of frontier foundation model providing? In the past, it was Meta for a long time with Llama. Before that, I remember Mistral was big. So one, I'm curious what you think happened to Mistral. Right now, it seems like Deepseek is all the rage. But who do you think will be kind of the research lab slash economic engine tied to it that will be driving a lot of the kind of frontier development on the open side? Question for any of you.

I think it's kind of hard to predict because there are a lot of factors involved. The magic of open source is that you basically need at least one player at any given time to release a good model to build upon. So I think the leader might change in this space, but the overall kind of market industry will continue.

I think what's common with Deepseek and the Qwen models from Alibaba and the Llama models from Meta is each of those companies is seeking to solve a problem that's not just distributing the model and selling it. You may have seen Meta's app they launched this week, which is a social network effectively. And from spending some time with the model builders in China, they are also looking to solve the next great consumer app or consumer use case. And so on the way to that, they're kind of just open sourcing these models. And so, you know, I think with Deepseek, what we happened to see was they happened to, along that path to that, no reason to keep these things closed source. They get more feedback if they open source it, that they can open source along the way, and they end up open sourcing something great. I think it was pretty difficult to predict the DeepSeek moment that happened in January.

Let's talk more about the DeepSeek moment. How did this happen? And, you know, we have Zuck throwing lots of compute at Llama. How did we get to the point where, you know, a Chinese model was top of the, top of the leaderboards, but also more importantly, top of developers' minds in the open source world?

Well, I mean, I feel like from Model itself, they did really good engineering. I think one thing which we heard also a lot from some previous presentations, small teams can move really fast and you kind of need to integrate research and engineering, systems engineering aspects to train a great LLM. And I think that's something that they did really well. And yeah, sometimes it's harder to do in bigger settings.

I think there was a combination of a couple of things. One is that it was a very good open-source reasoning model. We hadn't had one yet. Two, there was a UX innovation where you could see the thoughts come out of the model, which you could not do with, with O-1. And there were all sorts of just light bulb moments that the community had by doing that. And it also just felt more transparent. So it jived very well with the open source brand that DeepSeq's trying to build. And 3, they couldn't handle the inference. Their servers were just completely toast. You couldn't even pay them. They, like, blocked the payments on their— on their website. And so American companies were forced to figure out how to scale this up. And like Fireworks did an incredible job, captured most of the inference, I think, in the early times, as far as our metrics told us. And we saw not just like American companies get really interested, but also decentralized inference providers finally have a model that they could run that people actually wanted. And we saw a wild explosion there that I think most people in the AI space still aren't aware of.

Dima, can you give us, and not to turn this into a Fireworks advertisement, which we do not want to do, but give us a sense of what you guys did to optimize the model to make it work so well. And I'm curious some of the nuances of, I read that for example, Perplexity fine-tuned or post-tuned R1 to remove some of the, distinct elements to that model. Say more about anything you can say about that.

I mean, so first of all, on scaling, yeah, thanks for callout, Fadia. Basically on scaling, I mean, the model itself is like, it was first really big successful open source mixture of experts models. I mean, now Llama 4 is also a mixture of experts, like BigGAN, et cetera. With mixture of experts, there's a lot of interesting infrastructure problems, how to scale that well because you go into multi-GPU, multi-server distributed inference and that's something which we do really well. So there's a lot of interesting engineering which went to that just so your servers don't melt down and you don't have to turn down customers. And as you said, we saw a crazy growth from Christmas Day when it was released, maybe a few weeks for people to realize and catch up and we saw a lot of growth from existing customers and new customers in usage. Yeah, I mean to your question about distinctive elements, yeah, I think Perplexity and some other folks did a great job fine-tuning it for particular behaviors. We also saw a lot of other customers actually doing fine-tuning of DeepSig in particular for different tasks. So it's a really good pre-training-based model, and you can morph what you want to get into. And yeah, your previous point about fearmongering about that stuff, In many use cases, it doesn't apply directly. You actually want to focus on some particular business use case. This sensitive elements which people post online, they just don't come up in product questions. If somebody asks you that use model in that use case about politics, you don't actually want to get any answer. You want to stick it to narrow business use case and fine-tune in customization for particular business use cases works really well for those. For those hints.

Great. What are your thoughts on Llama 4, and where do you think it goes from here? Would you hypothesize that we end up with Llama leapfrogging back over DeepSeek?

There is a good chance for that, and we definitely hope so. But yeah, I think fundamentally all the ingredients are there. There is high-level commitment. There is amazing people work in meta and there's a lot of compute. So yeah, I think there are some things which maybe didn't go ideal in the current release and some of that might actually go with benchmarking. I mean, I see the public, generally benchmarking of LLMs right now is reaching the peak point. It's really hard to evaluate what is good and what is bad, but I think fundamental ingredients are there and hopeful for Llama Next.

Yeah, I think there's I mean, there was 3 when they announced it on Saturday morning. There was the behemoth and the 2 models they released, Scout and Maverick. And then this week Mark talked about, you know, little Llama or the smaller one, which this was the first release of Llama that didn't have that. And you know, a lot of the feedback at the event was that customers wanted that. I mean, that was one of the value propositions of Llama. And so I wouldn't count them out because they have the biggest model and then they have, the most usable, I guess, model that aren't out yet. And one of the things that really, that I think Deepseek did really well is they obviously had this massive model open-sourced and then they hosted it for a really cheap price, free for most consumers. But they also released these distillations. And I think that, you know, from a practical standpoint of adoption was really powerful. And I think, you know, what happened with Llama 4 was kind of the opposite of that. But I think in future releases could resemble that and it could be even much more powerful. Mark's got the ingredients, right, of a ton of resources, but also, you know, he's solving kind of this problem of how does he grow Meta's business with this, which may not be selling the model directly, it may be actually using it in their consumer applications. And so that feels a lot like the kind of ingredients we had for DeepSeq, but it's hard to know these things.

I'll just add that it's kind of all about learning rate for a lot of open source and, like, you know, a lot of learnings happen, I think, for Llama 4. It sounds like they're taking them into account. Reward hacking and, like, you know, like leaderboard-type benchmarks, I think, will have to adjust, and we'll have to get better at evaluating models to avoid reward hacking.

Okay, we're gonna end with one last question, and this is a quantitative question. I am not as Slido proficient as my colleague Constantine, but if you had to predict in 5 years' time What percentage of inference tokens will be running through closed versus open source models?

That's a really hard question for two reasons. I'd say 50/50 because there's a ton of great research on this. I think two great examples of that, one of them is RoutLLM that talks about using an open source model with a big model and switching between the two. There's the Minions project out of Stanford where it actually uses them in combination. And that primarily the workload runs on the small model. And so I think we're going to see a lot of work being done in the routing layer. So it's hard to know, but probably 50/50. That's my bet.

I mean, I think it's hard to get exact numbers, so I would probably go also with 50/50. But I think if you were to plot the most used models overall, I think on closed source, it might be still a few leading players with maybe a quarter overall going through Company X or whatever. I think on open source, it would be not a single model. It would be much more families and functions and customizations. And yeah, the earlier question about open source versus closed source, I think one scenario where if the gains were coming only from pre-training, I think actually closed source had advantage because you need to just keep building those trillion-dollar data centers or whatever. We heard a lot today about switching from pre-training to post-training and RL. And the interesting part there is that The recipe is kind of working and you can apply it to many problems and still if you want to try to build the model which is good for everything, you need to spend a lot of compute to train it in all the possible problems. But if you're taking the kind of customization heads, you can spend maybe less compute but use your unique data in your unique way of which problem you're actually trying to address. What are the right evals? What is the right problem formulation and RL environment formulation to get this maybe less capital investment to a really good result. And I think given this shift, I actually think that open source will grow overall as a fraction, even if there won't be necessarily a single model which will dominate everything.

More than 50%. Alex, what about you?

5 years is ages. I think If decentralized providers don't happen or can't happen sustainably, then I tend to be on the side of maybe closed source being more than majority until something like that happens. It's just my view right now. If providers figure out how to decentralize, then I think that changes. And I know like decentralization's not, you know, a hot part of the AI space right now, but I do think it completely changes the dynamics and it allows, it basically ties AI a little bit with like a financial system that kicks off a cold start flywheel and helps new people serve traffic. And they're like, all the existing players right now are, are very new. Some of them have wild incentives. Like, one that we route to is earning, like, $360K per day in incentives. And it's a big question just whether they're going to be sustainable.

OK, crypto guy bringing the economic engine to decentralization. Love it. Alex, Dima, Jeff, thank you so much for doing this today.

Thank you.