In this interview segment, Lin Qiao, CEO and Co‑Founder of Fireworks AI, outlines the most common generative AI applications he observes today. He groups them into three major buckets: (1) Assistance – AI‑powered tools that augment professionals in medicine, education, and law, helping address workforce shortages and boost productivity; (2) Chat‑based experiences – conversational agents that combine text, voice input, and voice output, enabling more natural interactions; and (3) API/Code Generation – middleware that lets developers programmatically generate business workflows, SQL queries, charts, observability dashboards, and other artifacts, often chaining multiple models or modalities. He emphasizes the disruptive, empowering nature of these applications and hints at the next discussion about moving generative AI from prototype to production.
Can you talk me through what are the most common uses of generative AI that you're seeing today?
Oh, that would be a— it would take 1 hour by itself. So yeah, so it is absolutely— this technology is so empowering and so disruptive. We have absolutely seen a lot of interesting, innovative applications and products built on top of our service tier. Of course, we also offer inference engine for GNI models. So I will kind of bucketize them in different ways. So one big bucket is sort of assistance. So we have seen people building medical assistants to address the shortage of doctors or nurses. And help those medical, medical workers become productive. We have been seeing people building, many people building educational systems, target different cohorts. For example, there are people building applications for students to learn, for researchers to do research, and for people to learn foreign languages. So it kind of cross-border, various different applications, education. There are also legal assistants where they find people to do case study and because kind of lawyers are expensive or help lawyer to become more productive. So I think that's a big chunk of application. And then we see a lot of chat. Just kind of interesting chat-based application. And usually those chat-based application combine model, combine voice with it. So it's not just, hey, you have to type, you can voice in, voice out. And those are also very popular. Another big bucket is kind of API generation or code generation. It's not for end consumers, it's for another program. So this is important because we are seeing the compounding effects of people building application on top of not just single model, but multiple models or multiple modalities, or from multiple modalities to additional APIs. So there are example, the totality of knowledge I can drill down more to that because single model has limited, very limited knowledge by itself. The specific use cases are generating business workflows, generating SQLs, generating charts about the production usage, monitoring observability, and so on. That's a huge bucket. Those are across the board where we're super excited of what's happening.
I feel like if you continued, you could probably just listed every industry that there is. But that's really interesting that you got stuff that is blind to end users. You mentioned things like productivity improvements and training. Then you've also got that sort of middleware layer. So it's for other people building technology as well. OK, so since today we're going to talk about getting things into products and into production, I'd like to know just at a high level, what are the different steps for getting generative AI into a product?
First of all, I would say application built on top of GenAI on GenAI now or application pre-GenAI, there are a few differences. One is pre-GenAI application built on top of CPUs, right? It's heavily commoditized. The stack, software stack is very mature and the tools are kind of abundant, so it's kind of easy and low cost. With GenAI shifted, right? GPU is new to most of people, and the GPU is very expensive. GPU is power hungry, power expensive. Power generates heat, heat requires cooling, cooling is expensive. So the whole cost structure of productionized application on top of GenAI significantly changed right now. That's one. And the second, Between, like, before and after, whether you're building on GNI or not, one thing that doesn't change is most of those application is consumer-facing, is developer-facing. It need to be very interactive. So latency is a critical part of product experience. Holding the latency bar doesn't change before and after. Another interesting thing that is different is before GenAI, the logic, the reasoning logic is coded in the application itself, right? Or it's coded in the surrounding infrastructure, so it's deterministic and you can debug and the logic is there. With GenAI, the GenAI models have multiple layers and with multiple layers of prediction and it's probabilistic. So probabilistic means if you ask the generative model to produce something, it will do it regardless whether it's true or not. So that directly introduced a new phenomenon, hallucination, which application developers never need to deal with. Now they're like scratching their head like, oh, this is bad, and I need to learn how to control or minimize hallucination. In my product, in my application. So those are the kind of one invariant and two variants that's happening in kind of pre-GenAI and current GenAI era. And because of that, that introduced new production challenges for our app developers. For the invariant, low latency, right? As we know, the GenAI models is on the extremely big side of machine learning models. And because the model is so big, and the latency will be high, and like not having a low latency product experience, it will be break or make it scenario for the app developers. And so that is extremely important, we need to address that. And the cost is also important factor. Even if you have a viable product, if the cost is very high, and the nature of consumer-facing, developer-facing apps, they scale quickly if you hit the product market fit. When you scale quickly, you're losing money at a small scale, you're gonna bankrupt at a large scale. So controlling cost is extremely important to go into production. And also to produce great product quality and content, You have to control hallucination, and learning the skill set how to do that, extremely important. So I think those are the kind of 3 critical parts to bring GenAI technology into production.
