Lin Qiao, CEO of Fireworks AI, breaks down the three types of GenAI applications gaining traction: productivity tools for medicine and law, conversational interfaces with voice, and API services that automate workflows and generate code. He notes that the best solutions stack multiple models together to overcome what any single model can't do alone.
Can you talk me through what are the most common uses of generative AI that you're seeing today?
Oh, that would be a. It would take one hour myself. So yeah, so it is absolutely. This technology is so empowering and so disruptive. We have absolutely see a lot of interesting innovative applications and products built on top of our service tier. Of course firewalls offer inference engine for JNI models. So I will kind of bucketize them in different ways. So one big bucket is sort of assistance. So we have seen people building medical assistance to address the shortage of doctors or nurses and help those medical workers become productive. We have been seeing people building, many people building educational assistance target different cohort. For example, there are people building applications for students to learn, for researchers to do research and for people to learn foreign languages. So it kind of cross border various different applications education. There are also legal assistance where they find people to do case study because kind of lawyers are expensive or help lawyer to become more productive. So I think that's a big chunk of application. And then we see a lot of chat, just kind of interesting chat based application and usually those chat based application combine model combine voice with it. So it's not just hey, you have to type, you can voice in, voice out. And those are also very popular. Another big bucket is kind of API generation or code generation. It's not for end consumers, it's for another program. So this is important because we are seeing the compounding effects of people building application on top of not just single model, but multiple models or multiple modalities or from multiple modalities to additional APIs. So there are example, the totality of knowledge. I can drill down more to that because single model has limited, very limited knowledge by itself. And the specific use case are like generating business workflows, generating SQLs, generating charts about the production usage, monitoring, observability and so on. So that's a huge bucket. Yeah, so those are kind of across the board where we're super excited of what's happening.
Okay, I feel like if you continued you could probably just listed like every industry that there is. But that's really interesting that you got stuff that is blank to end users. So you mentioned things like productivity improvements and training. Then you've also got that sort of middleware layer. So it's for other people building technology as well. Okay, so since today we're going to talk about getting things into products and into production, I'd like to know just at a high level, what are the different steps for getting generative AI into a Product.
First of all, I would say application built on top of Genai on Genai now or application pre gen There are a few differences. One is pregenai application built on top of CPUs. It's heavily commoditized. The stack software stack is very mature and the tools are kind of abundant. So it's kind of easy and low cost. With Geni shifted, right? GPU is new to most of people and the GPU is very expensive. GPU is power hungry power expensive power generate heat, heat requires cooling, cooling is expensive. So the whole cost structure of productionized application on top of Genai significant changed. Right now that's one. And second the between like before and after whether you're building on DNI or not. One thing that doesn't change is most of those application is consumer facing, is developer facing. It needs to be very interactive. So latency is a critical part of product experience. Holding the latency bar doesn't change before and after. And another interesting thing that is different is before Genai the logic, the reasoning logic is coded in the application itself, right? Or is coded in the surrounding infrastructure. So it's deterministic and you can debug and the logic is there. With Genai the Geni models have multiple layers and with multiple layers of prediction and it's probabilistic. So probabilistic means if you ask the general model to produce something, you will do it regardless whether it's true or not. So that directly introduce a new phenomenon, pollucination which application developer never need to deal with. Now they are scratching their head like oh this is bad and I need to learn how to control or minimize hallucination in my product, in my application. So those are the kind of one environment and two variants that's happening in pre Geni and current Geni era because of that that introduced new production challenges for our app developers. For the invariant low latency. As we know the general models is on the extremely big side of machine learning models. Because the model is so big and the latency will be high and not having a low latency product experience will be break or make it scenario for the app developers that is extremely important. We need to address that. The cost is also important factor. Even if you have a viable product, if the cost is very high and the nature of consumer facing developer facing apps they scale quickly. If you hit a product market bid when you scale quickly, you're losing money at a small scale, you're going to bankrupt at a large scale. So controlling cost extremely important to go into production and also to produce great product quality and content, you have to control hallucination and learning the skill set. How to do that, extremely important. So I think those are the kind of three critical parts to bring Genai technology into production.
