Lin Qiao explains how enterprises will build specialized AI models using Fireworks' reinforcement fine-tuning. A modest open-source model can become frontier-level on a specific task in a month, but gains expertise in one domain at the cost of losing capability elsewhere, like human specialization.
Chapters
You see the future as millions of different models. These could be like LLMs, but millions of different AI models that are tuned to specific tasks within specific enterprises. And it sounds like Fireworks offers a reinforcement fine-tuning product that allows your customers to beat frontier closed models in under a month on specific tasks that that relatively small tuned open model is fine-tuned for.
Right. So we are very bullish in this direction. So think about AI has been following a lot of footsteps of human intelligence, right? So artificial intelligence has been following that footstep, even the model architecture is called neural networks. It's kind of emulating like the human, human's brain, right? So for reinforcement learning, it is actually very similar to we, how we human learn knowledges. So we learn by various different angles. So one of the angle is we get positive feedback that we know, oh, this is the the correct thing to do by learning the principles, or we get negative feedback and it's kind of we get penalized for doing something bad from our behavior point of view. Then we learn, no, don't do it anymore, and we kind of change to a different direction. So this also happened to how we do agriculture, for example. As you know, I'm a fruit lover. And, you know, today we all like to eat like sweet, sweet fruit, very— the fruit also grows much bigger, but this is not how originally it became, right? It gone through multiple generations of selection process where we collect the seeds of the sweeter and bigger fruits in the planet and among those and collect another. So this is kind of another way of reinforcement learning, reinforcement seed collection and selection process. So a lot of things we do, whether for ourselves, our own learning process, or we have applied in other domains, is following the same principle. And this is also similar to how models learn. You teach a model what is a positive reward, you teach a model what is a negative reward, and the model is going to automatically scan through a search space of possibilities and through these feedbacks, and find a path to specialize itself in solving certain kind of problems really, really well. With that said, it's not all like only benefits, right? It's a trade-off because if you let the model specialize in certain direction, the area, it's going to be less specialized in other areas. So it, it's like us exactly. As John, you specializing in driving this podcast and you're a great host and you You're very knowledgeable how to engage with guests. And I specialize in building the best AI platform, which can customize towards application-specific patterns and so on, right? So, but we then, I'm not a good chef or cook. And I don't know actually how to you know, how to do gardening very well. So see, so that's kind of a natural selection we have, direct our attention to similar to the models. So that's kind of where we are betting on this direction to make reinforcement learning for models very, very accessible to all application developers so they can basically have a model in tune to their product all the time. And they imagine that, imagine a model is just constantly learning the intelligence from your application, and then you have your private model, and then you have your mode where nobody else built on top of an existing API would have. So this is a special thing that we want to kind of have every application developers to get hold of.
All right, I see. So all of that private data that you mentioned at the outset of the episode, all of— that accounts for most of the data in the world, enterprises can be using their particular private data to be creating a moat by not only having that private data, but also having these fine-tuned models specifically specializing in particular aspects of their data applied to specific tasks.
Yep. And this is not a new thing, actually. For example, during the mobile-first move, so before AI, the biggest shift is mobile. So the application moves from desktop to mobile devices, and that actually opens up a whole new domain of doing product development, which is similar to autonomous intelligence we're heading towards in, in terms of thinking. [Speaker:LIN QIAO] On mobile first, I think one thing that opens up is the access of end consumers to an application, right? Because the people who owns desktop versus people who owns a phone, it is orders of magnitude different. And that just means now your product could have reached to orders of magnitude higher group of people. And it will go global much quickly. And you will be able to access various different demographics of the cohort much wider. It changed how app developers think about product, product evaluation, because now it's much broader. And you cannot just deploy a group of PMs to understand what the customer wants. And the product no longer becomes monopoly, just one design. Because for different cohort coverage, you may want to highlight one feature versus the other. So, so then it becomes, oh, how do we even do product development? How do we incorporate that feedback? It has to be customized. And that customization is being done through a new technology called A/B testing, right? So the idea here is to use the insights from your product to compare A versus B, of the product feature and make statistical decision based on statistical result, make the decision where your product should look like within certain cohort of groups. So the idea is basically similar, like product has a lot of intelligence. We need to leverage that intelligence to make the product better. And here, similarly, product has a lot of intelligence. We need to leverage intelligence to make the model better for your product, therefore your product built on top of specialized model will be better. So right now, it's a completely open space, it is a vacuum. And there's no existing solution that has been solving this problem really well. I'm pretty sure industry will move towards looking at this space very closely. And I firmly believe there's a lot of value, uh, in there.
