971: 90% of The World’s Data is Private; Lin Qiao’s Fireworks AI is Unlocking It

Over 90% of the world's intelligence is locked inside private enterprise data that no foundation model has ever seen. Today's guest is on a mission to unlock it. Welcome to episode number 971 of the Super Data Science Podcast. I'm your host, John Krohn. Today's guest, Lin Qiao, is the CEO of Fireworks AI, a Bay Area startup that has raised over $300 million to unlock the world's vast quantities of enterprise data for LLM training and inference revolutionizing capabilities and performance. With a PhD in computer science from UC Santa Barbara and years of experience as a director of engineering at Meta, Lin is now a highly successful technical founder with a rich perspective on AI today and what the future holds for all of us. Enjoy this one. This episode of Super Data Science is made possible by Dell, Intel, Cisco, and Exceldata. Lin, welcome to the Super Data Science Podcast. It's an honor to have you take time out of your busy schedule to be on the show. How you doing today?

I'm doing great. Thanks for having me, John.

Of course. And where are you calling in from?

I calling from Portola Valley. That's where I live.

Nice. And that's—

It's part of Bay Area.

Yeah, yeah, yeah. Palo Alto.

Yeah, very close to Stanford.

Nice, nice, nice. So we're here to talk about Fireworks AI, your business. Which has done incredibly well. I mean, you've just grown so quickly. I believe you've now raised over $300 million in venture capital, including a recent $250 million Series C, if I got that correctly.

Right.

And so it's a platform built around open-source model deployment at scale. And the Fireworks AI platform is built around open-source model deployment at scale and this idea of autonomous intelligence. Tell us what that means, Lin.

Yeah, sure. You're right. We raised our last round last year and we are growing really fast. So our mission is autonomous intelligence. This mission is very complementary to AGI, where the direction of AGI focuses on investing in directing a lot of intelligence into this one model. And have this model be able to solve various different kinds of tasks in a great way. So the idea is you just build your application on top of the AGI model as a utility, right? So this is a great— AGI is a great direction. It's very scalable if it's successful. But the reality is only a very small fraction of data goes into the foundation models for AGI. If you look at the world's data, majority of the data, by majority I really mean like more than 90% of the data, is actually not in the public domain. It's not in public internet. It's not labeled by labeling companies, which goes into the foundation models. And majority of those data are private data locked inside applications in enterprises. And data, we all know data is intelligence, data is knowledge, and those application-specific, enterprise-specific data is not accessible by the AGI labs. And we just leave a lot of intelligence on the table. My prediction, and that's where we're betting on the future, is to be able to activate those private data and let the model absorb additional application-specific intelligence and bring the model to the next level. And this kind of motion is more like customization, right? It's kind of— it is the model and the inference deployment were customized towards applications in their specific pattern. And this customization should not be just one time. Right? Our application enterprise product, it keeps evolving. So this customization should be continuous. And ideally, this continuous customization should be fully automated. That's autonomous intelligence. We are making great progress towards that direction. And we believe the future is not one model rules all. It's going to be millions of models, one per application. Per use case.

All right. So the term autonomous intelligence, it sounds kind of vaguely to me like the buzzword of 2025 in our field, probably the buzzword for 2026, which is agentic AI. But it sounds like it's quite different from agentic AI, this idea of autonomous intelligence.

It's heavily connected with agentic AI. So think about agent as a way to automate many of our day-to-day tasks, right? So We have been living in a world that many expert-intense tasks have been gradually automated so we can free up our time. Eventually, some of the even professions will be redefined. So for example, I think there are interview agents or hiring agents where you give a job listing, it will source the candidates and even do the first rounds of filtering and interview for you. And there's marketing agent, you give your ICP list and we'll source the right company, right stakeholders, and start to drive customized outbound emails and reaches. And there's customer service agent, just give the human agents some really good assistant, assistants who kind of be smart. And there are so many like agents to doctors and so on. So this is happening, it's transforming our day-to-day life. But similarly, another big transformation that's happening is in my domain is software development is being disrupted. And today, a coding agent can really start to behave like a junior engineer. And I'm not kidding, this is kind of really happening. And it's actually changed our interview process. And The fundamental question we are asking ourselves is the coding, is coding, coding interview important anymore, right? So coding interview in the, in the past is gonna replace by, you know, how good you are at using coding agents. So, so it is actually happening across our day-to-day life. Now let's go back to, to this, Autonomous intelligence era, right? So without that, currently this work of continuously adapting the model and changing and customizing inference setup is done by a very, very small set of experts. So those experts are like, they have been doing AI system for a long time. They have been researcher for a long time. Accumulated knowledge over years. So only a few companies who has those strong density talent pool are able to do that. And the question is, can that part be automated, right? And similar to other areas that has been disrupted and has been reshaped, and can this part of doing product model co-design And, you know, infusing more intelligence in the model and making the inference serving tier much faster and much more efficient. You know, can that part be accessible by a wide range of application developers without them putting in a lot of work and carrying the burden of learning all the deep knowledges? So that's what that means. So if we heard a lot about AI is going to free up a lot of human labor, right? So, and this wave is interesting because it will start from a different angle. It will actually free up a human from the high intelligence level, not from the physical level, right? The robotics is going to disrupt the physical level. Of engagement, but AI is going to free up a lot of kind of high intelligence level of the task as work. So that's kind of interesting change. And we are also innovating and disrupting in that space from a baseline platform space.

Really cool. So it sounds like autonomous intelligence builds on agentic things, but also involves lots of things not associated with agentic, like systems, like models automatically retraining. and the whole kind of system humming along nicely. Um, it seems like a key part of that working for you, especially in an earlier answer, you mentioned how you see the future as millions of different models. These could be like LLMs, but millions of different AI models that are tuned to specific tasks within specific enterprises. And it sounds like Fireworks offers a reinforcement fine-tuning product that allows your customers to beat frontier closed models in under a month on specific tasks that that relatively small tuned open model is fine-tuned for.

Right. So we are very bullish in this direction. So Think About AI has been following a lot of footsteps of of human intelligence, right? So it's artificial intelligence has been following that full step. Even the model architecture is called neural networks. It's kind of emulating like the human, human's brain, right? So for reinforcement learning, it is actually very similar to we, how we human learn knowledges. So we learn by various different angles. So one of the angle is We get positive feedback that we know, oh, this is the correct thing to do by learning the principles, or we get negative feedback. And it's kind of, we get penalized for doing something bad from our behavioral point of view. Then we learn, no, don't do it anymore. And we kind of change to a different direction. So this also happened to how we do agriculture, for example, as you know, I'm a fruit lover. And you know, like today we all like to eat like sweet, sweet fruit. Very, the fruit also grows much bigger, but this is not how originally it became, right? It gone through multiple generations of selection process where we collect the seeds of the sweeter and bigger fruits in the planet and among those and collect another. So this is kind of another way of reinforcement learning. Reinforcement, seed collection, selection process. So a lot of, lot of things we do, whether for ourself, our own learning process, or we have applied in other domains, is following the same principle. And this is also similar to how models learn. You teach a model what is a positive reward, you teach a model what is a negative reward, and the model is going to automatically scan through a search space of possibility. And through these feedbacks and find a path to specialize itself in solving certain kind of problems really, really well. With that said, it's not all like only benefits, right? It's a trade-off because if you let the model specialize in certain direction, the area, it's gonna be less specialized in other areas. So it, it's like us exactly. As John, you specializing in driving this podcast and you're a great host and you, You're very knowledgeable how to engage with guests. And I specialize in building the best AI platform, which can customize towards application-specific patterns, and so on, right? So, but we then, I'm not a good chef or cook. And I don't know actually how to you know, how to do gardening very well. So see, so that's kind of a natural selection we have direct our attention to similar to the models. So that's kind of where we are betting on this direction to make reinforcement learning for models very, very accessible to all application developers so they can basically have a model in tune to their product all the time. And then imagine that, imagine a model is just constantly learning the intelligence from your application, and then you have your private model, and then you have your mode where nobody else built on top of an existing API would have. So this is a special thing that we want to kind of have every application developers to get ahold of.

In data science, the right hardware is essential for performance. This episode of Super Data Science is sponsored by Dell Technologies and Intel. Dell's latest AI PCs and workstations powered by Intel Core Ultra processors feature dedicated AI acceleration on the chip. This delivers faster inferencing, smoother multitasking, and longer battery life, even with demanding local models. Your hardware won't hold you back Whether you're prototyping in PyTorch or pushing production workloads, see how Dell AI PCs with Intel Core Ultra fit your workflow at dell.com/shoppcs. That's dell.com/shoppcs. Dell and Intel, designed for how data scientists really work. All right, I see. So all of that private data that you mentioned at the outset of the episode, all of— that accounts for most of the data in the world, enterprises can be using their particular private data to be creating a moat by not only having that private data, but also having these fine-tuned models specifically specializing in particular aspects of their data applied to specific tasks.

Yep. And this is not a new thing, actually. For example, during the mobile-first move, so before AI, the biggest shift is mobile. So the application moves from desktop to mobile devices. And that actually opens up a whole new domain of doing product development, which is similar to autonomous intelligence we're heading towards in, in terms of thinking. On mobile first, I think one thing that opens up is the access of end consumers to an application, right? Because the people who owns desktop versus people who owns a phone, it is orders of magnitude different. And that just means now your product could have reach to orders of magnitude higher group of people. It will go global much quickly, and you will be able to access various different demographics of the cohort much wider. It changed how app developers think about product, product evaluation, because now it's much broader. And you cannot just deploy a group of PMs to understand what the customer wants. And the product no longer becomes monopoly, just one design, because for different cohort coverage, you may want to highlight one feature versus the other. So, so then it becomes, oh, how do we even do product development? How do we incorporate that feedback? It has to be customized. And that customization is being done through a new technology called A/B testing, right? So the idea here is to use the insights from your product to compare A versus B of the product feature and make statistical decision based on statistical result to make the decision where your product should look like within certain cohort of groups. So the idea is basically similar, like product has a lot of intelligence. We need to leverage that intelligence to make the product better. And here, similarly, product has a lot of intelligence. We need to leverage intelligence to make the model better. For your product, therefore your product built on top of specialized model will be better. So right now, it's a completely open space, it is a vacuum. And there's no existing solution that has been solving this problem really well. I'm pretty sure industry will move towards looking at this space very closely. And I firmly believe there's a lot of value in there.

Awesome, I love it. Yeah, really exciting. You're kind of defining a new category, uh, which is an exciting thing to be doing. Um, so when people are trying to figure out these, you know, these relatively small, say, LLMs for a particular task, does model selection really matter? I mean, do your— are your clients like picking, oh, okay, I'm going to use this Llama model of this size, or Qwen, or do they make those decisions, or is something that's kind of like handled automatically by the autonomous intelligence system?

So model selection matters, but it's also exhausting. So funny thing, um, there are two interesting phenomena that's so unique to AI. One is the model depreciation is very fast. As you can observe, every couple of weeks there's a new model launch. While it's closed open, someone topped the leaderboard, and then a couple of weeks later, someone else topped the leaderboard. And we often— they all are strong in different ways. So, and we start to see the researcher focus start to diverge. Right. It's clear some labs are really good at chat-based models. Some labs are really good at coding to use agentic models. Some labs are really good at multimodality models. Some are really good at long context. They'll all start to diverging to focus on specializing in different areas. Speaking about specialization. So different use case will suit different kind of model specialty very well, but it's really hard for people to figure out which one is the best for my use case. You know, my use case will also evolve over time and the benchmark result, public benchmark result is fully saturated and we need to figure out how to pick and choose continuously and which is exhausting. So we do help our customer figure that out. But at the same time, there's another level of complexity is hardware depreciation is also very fast. This is something new. Before this wave, usually every 3 years there's a new hardware SKU. And now last year myself, NVIDIA launches 3 SKUs, 3 new SKUs, and 2026, there will be a lot more new SKUs from all different vendors, whether it's GPU or custom ASIC. So then how to manage hardware becomes a very hard problem. And these two combined is causing so much headache to the application developer who want to stay on top of these, all different kind of wave. It's just too much deep knowledge and expertise they need to gain. In order to kind of pick the best for them. So we are the platform that we abstract out, we eventually want to abstract out hardware. We want to abstract like hardware selection and we want to kind of provide the best model for various different use cases. And so basically there are various different kind of mapping from specific use case, specific workload patterns to the model, to the hardware, to the inference setup, to the fleet design. So all of that requires a big team in-house with deep experts, which is really hard to find. And we want to kind of make it super easy for our customers.

Right. So instead of needing to find the deep experts, they can just come to Fireworks and work with you guys, work with your solutions.

Absolutely. And interestingly, there are— so last year by itself, I haven't counted. There are every month, there are a few new models get deployed and launched. And it has been very exciting. But also, I will say early on, we bet on open models. That was a big debate in the company that We debate on many what-ifs, right? But because our background in PyTorch, which is an open source project— PyTorch is now— we built PyTorch from ground up. It is now the dominant AI framework. We firmly believe in the power of open science, and it will, in the long run, we believe that is going to really shape the industry in an unprecedented way. So that's why we bet on open models from very early on. And it's great to see that open model performance is converging with closed model. I think 2026 is the year that that convergence will become more prominent and super excited about that.

Yeah, something that I haven't mentioned on air yet is that for 7 years before founding and being CEO of Fireworks AI, You were at Meta as a senior director of engineering where you led over 300 engineers, and a big part of what you were doing there, based on what I could find online, is developing PyTorch. So thank you for that.

This is definitely a very big team effort, and I have a very talented team at that time. And yeah, so I'm very thrilled about the industry-wide impact of PyTorch. There are many great researchers and leaders, engineers I'm lucky to work very closely with. And some of them start a company with me, Fireworks. So really great crew, and we continue to going down the same path of democratize AI to the whole entire industry.

Really cool. All right, so back to Fireworks and what you were saying about open source, not just PyTorch, but open source models and how you at Fireworks have made this bet to embrace open-source AI models. Do you think that a lot of organizations, a lot of enterprises underestimate what's possible with open models?

I don't think it's about underestimation that much. It's just about not being familiar with open models. I think especially, I think I will put, simplify this answer. There's two groups. There's AI-native startups, there is enterprises. Obviously enterprise has like digital natives and traditional enterprise and various different categories. But let's just take a look at these two groups. Their reaction is different. AI-native startups are, they just want the try the best option and they do not have any baggage. They're very, very brave in testing open models. They're always very curious. And at the same time, there's a very interesting phenomenon that's also unique to AI, right? So as you know, there are many companies, whether startups or incumbents, They're trying and experimenting with new user experiences that our day-to-day is interacting with. And that, those kind of experiments has been, have seen a lot of success in terms of product market fit. But product market fit doesn't, at AI time, doesn't mean a viable business. There could be a big 10, that people are willing to pay the product, pay for the product, but the cost of running the business could be much higher than the revenue you get. And then people just cannot scale their business after they hit the product-market fit. And it's funny, literally they're going to scale into bankruptcy. So obviously startups are like, have limited funding and even including incumbents, they cannot— they have the distribution and they have to be so careful. The CFOs have to be so careful in controlling, uh, the cost. So, so the kind of budget and this ROI totally makes sense. And because of that, they have to hold a lot of people on the waiting list and cannot open up the gate. So, um, while the startup— going back to AI-native startups— they are very brave in embracing open models. But part of motivation is open models are, they have a lot more control. They can customize however they want, but also at the same time, open models are much more economical because we as a provider, we do not have pre-training cost, billions of dollars pre-training cost amortized over. And we only focus on bringing the best quality, speed, and cost, like from post-training, which is much cheaper, and inference customization of the deployment. So the unit of economics of Fireworks operating open model is very different from Frontier Labs. So that's kind of where we see the AI-native startups are very brave in adopting and going down the path of customization with open models. The second group of enterprises, obviously they're much more careful because many of those are public companies and they are under certain kind of obligation, especially their legal team to understand what are the implication of using open models license, what are license limitation, just kind of trying to understand What is this beast? So, but I, from my observation, the enterprise start to open up and they start to understand much better, oh, this model actually is not going to send data to other countries. It's just a model. And depend if the model provider is hosting the model in US and then they can provide privacy and security around it. Around the ins and outs. So there, but anyway, there are other concerns as in the kind of content generated. Does that follow certain kind of guardrail and so on. So, but overall I've seen the whole entire industry start to understand better and start to kind of understand the bounds of operating open models. And I only see an upper trend of embracing open models.

Quick reality check for anyone building with AI agents. Your agents can discover each other, they can pass messages, they can coordinate on tasks. But here's what they can't do. They can't think together. When your agent figures out how to handle a complex workflow, that knowledge stays isolated. The industry has focused on scaling AI vertically, bigger models, more compute. Those breakthroughs matter, but intelligence also scales horizontally. Agents sharing knowledge across a network, coordinating on common intent, reasoning together. The infrastructure for that second horizontal axis doesn't exist yet. Outshift by Cisco is formalizing it. They call it the Internet of Cognition. They're publishing the architecture and building reference implementations. Read Scaling Out Superintelligence. We've got a link to that in the show notes. Then check out episode number 961. In it, Dr. Vijay Pandy the head of Outshift by Cisco, walks through how horizontal scaling of intelligence works and why it matters. Nice. That was a very cool explanation of those kind of two customer types, the AI-native startup, the enterprise. It sounds like regardless of which category a customer of yours falls into, a huge advantage of Fireworks must be that they can not only are running open-source models cheaper, like you said, but by running them through you, it means that they don't need to themselves, right, be buying the GPUs, be managing all the ops around that— physical ops, software ops. They just can rely on Fireworks to get things up and running, and then, you know, having the right GPUs running behind the scenes, uh, you know, and controlling those kinds of costs is handled automatically.

So, at the beginning of this conversation, I mentioned AI is changing a lot of things in our day-to-day. I think one fundamental thing it's changing is the velocity of product development on AI is extremely fast, including the enterprises, right? So, because of that, it's really hard for enterprise internally to forecast how many, how much traffic this product is going to generate. It may generate no traffic because the experiment doesn't go to the phase they can go into production, or it can generate massive traffic. It's kind of the variation is very broad in this fast evolution of product development. So then it caused a big problem because without like a steady prediction of how to do capacity, like from finance point of view, and then what does that even mean to procure hardware? And you either over-provision or you either, or you under-provision. It's kind of the range is very broad. So that's where we can help because we're aggregator, we aggregate demand across all different customers and we take that risk off the table from being, you know, you want to worry about that and we're very flexible in accommodating varying requests because we can kind of add them together and drive the adoption. So I think that's kind of a unique unique nature of AI, but it's not that unique. But think about that, right? So in the early days of cloud-first, before cloud-first, everyone, every big enterprise, they have mainframe. And so data center locked in with keys in a cage of machines, right? This is the most rigid way of doing capacity planning. And the cloud, when cloud comes into the picture, Usually people think you are crazy. Why would you move from the most secure enclosure deployment of your infrastructure into renting someone else's infrastructure and you run your most important application there? But guess what? That cloud infrastructure provides so much flexibility that over the long term, it's much more efficient to run. So similar things is happening in AI space, and that's why it's very— it's fascinating. The velocity of AI development is shifting and changing and even disrupting how we do business in the traditional way.

Really cool answer again from you there. Thank you so much. I learned— I've learned so much from every response. And so it seems like you might have an interesting and informative perspective on reasoning models or slow-thinking models. So that's been a big thing last year. OpenAI, Anthropic, Google— there's a big fuss around the slow-thinking models that they released, and particularly their performance on complex tasks like Math Olympiads and writing academic papers and these kinds of things that require, you know, more processing before just outputting some kind of train of thought. Uh, I noticed on the Fireworks website that you mentioned very fast latency, like sub-2-second or even sub-500-millisecond latency in a lot of real-world deployments. Do you think that the, like, slower reasoning models that take time before they output something, do you think that there's a lot of enterprise need for that?

So I would classify, again, extremely simplified way of looking at application. So one type of application is real-time response. So it's either human-facing interactive response latency, or it's kind of fast transaction-facing, for example, fraud detection, right? So in aggregate way is kind of very fast going. The other type of application is offline. So for example, I have a case study, legal case study, I need my legal assistant to go off and analyze similar cases and come back in 3 days, give me a report. Now, an agent, legal agent can go off and do this study for a couple hours and come back, right? So those are 2 very big categories of agents or agent applications that has different latency requirements. So typically for the real-time, extremely low latency requirement, you cannot afford to think. You have to basically react. And usually the customization, the go back goes back to customization, right? Usually the customization process is very interesting also, is you go small, big, and then small. For real-time use cases. What does that mean is when you customize, a lot of time it's about your data. Is your data high quality? Keep in mind this kind of garbage in, garbage out, same, same apply here. If your data is not high quality, then your end model is not high quality. So for you to test data quality and to fix quality issue, you start with small model to see if the model is going the right trajectory. And then if it is not going, you go back to fix our data and then, you know, small model help you iterate fast. But that small model is not important. The trajectory of the model quality change is important. Once you like the trajectory, and then you move to the biggest model. And usually those are biggest MoE model, really hard to get it running. So obviously that's where we will definitely help you. And use the clean data to tune the model, get the best specialized model to solve your real-time problem. But usually those largest models is not as fast, right? Because they're very big. They're trillion, trillions of parameters, very big, but very good in terms of quality. And then you go small again, but you still find the biggest model you have tuned. Into a small model because you need a fast response. And then you launch the smallest model, you can get to high quality from that process and can go from there. So small, big, small, right? Now we move to offline asynchronous agents. And usually those agents are doing some deep research in certain kind of domain. Whether it's about legal, whether it's about finance, whether it's about software engineering, or whether it's about something else, right? So, but usually it takes time to think like us human being, we're gonna, hey, do some homework, go hide in a cave, figure it out. So similarly, and those model requires the highest level of intelligence, highest level. So then the kind of slow thinking mode becomes extremely important. But not just that, those big models for offline research also requires a lot more context, a lot more context to make the right thinking process, like get the right design of the flow. And, but at the same time, A lot of the time those models also need to get specialized in solving certain kind of problem really well. The legal process, uh, the finance process, the process of working, building PowerPoint, uh, the, the process of, um, building an executive overview, um, a pitch, uh, to investors, and the process of doing customer service, those are all very, very different. So we have also seen a lot of application agentic developers, they start to customize offline agents where that just means they need to figure out how to tune with the thinking tokens. So that's kind of a very different classes of equation I oversimplify here.

Oh, I mean oversimplifying, but making it very easy to understand as you have with all of your responses in this episode. I love particularly the small-big-small approach to finding the right model for your use case and having that be performant in real time. As you were discussing that, something that came to mind for me is how one of the most difficult things with deploying such complex models that have stochastic outputs is evals. And so even like when you're doing that small-big-small, like how do you ensure that when you go, say, from the big to the final distilled small model, you're still getting the same kind of performance. Eval can be so difficult. Do you have thoughts on that, or does Fireworks have any tooling that helps out?

Yeah, so for evals, we do not offer eval product. We instead partner with the companies whose specialty is doing eval. So here again, we really respect specialty and the customization and the focus. So you're right, eval is where people get started, right? So, but it's interesting if you talk to the startups or people building AI-native agents, there are so many different ways to do evals. First and foremost, people do vibe evaling. If you build application, the first thing you do, it's very different from software development practices. And usually when we write a piece of software, we start from unit test first, right? So, and then we build different kind of guardrail to make sure the quality, we have guarantees on quality, right? From various different unit tests to integration tests to then testing production, you kind of collect some metrics and so on. But in generative development, people usually start with some hypothesis and just look at result and do vibe testing. It's very interesting because they don't, they do not want to bound their creativity and the imagination by forming small test to big test. They test big ideas. So then, but those kind of vibe testing is really hard to harden. And drive further model customization because it's very subjective. And then the question is how you turn vibe testing into something more concrete that a model can understand or a tuning platform can understand. So I think it is a very important area and where we have been contributing is to solve another complexity. So today there are various different tuning platforms and the various different evaluation platform. As you know, the evals are directly feeding into tuning to drive the tuning process. Without eval, you basically don't have a guide. hey, this direction is correct or not, right? So, but there are so many different eval systems. There are different tuning systems, and we are part of the tuning systems, and there's no standard across them. The integration of cross-product complexity is very high. So we open-sourced a project called Eval Protocol. The idea here is to standardize the format so any eval system can talk with any tuning system. So for our developers, they, if they can pick any combination as they want, but because of Evol Protocol, it kind of bridge the gap across and they can, they have optionalities on both sides. So that's our intention to help bring more commonality across these two sides because these two sides are deeply integrated.

This episode is brought to you by Acceldata, the leader in agentic data management. If you've ever wished your data pipelines could think, diagnose, and even fix themselves, you'll want to check out StudioLab, Acceldata's new fully interactive sandbox. For 15 days, you get hands-on access to an AI-powered environment where you can explore data reliability, governance, and automation with real agentic workflows. No setup, no overhead, just instant impact. Experience how AI can transform your data operations. Start your free Studio Lab trial at acceldata.io/free-trial. That's acceldata.io/free-trial. We've got a link for you in the show notes. Cool. So as you were talking there, eval protocol from Fireworks hadn't showed up actually in our research, but I quickly found the GitHub repo for it. So I'll be sure to include that GitHub repo in the show notes. A different offering of yours that did show up in our research was something called 3DFIRE Optimizer. And so this seems like it's in a way, it's a way it goes over 100,000 possible ways of optimizing an LLM stack. Do you want to tell us about that?

Yeah, definitely. So when it comes to customizing, it goes back to our mission of autonomous intelligence. We are customizing across 3 dimensions, across model quality, model speed, and the model efficiency, which is cost. So when we started our journey, people were sure, hey, they're really concerned about cost, or hey, they're really concerned about speed because they are very interactive, or hey, they're very concerned about quality. Wherever they start their pain points, at the end, they are concerned about all three dimensions. There's no time where like, hey, the cost is really good here. We are like 10 times cheaper compared to alternatives. And they're like, go for it. They're always like, oh, we also need better quality to be able to launch this. Oh, we need it to be faster to launch this. Always, always all three dimensions has to be much better. So we're like, hey, this is actually not a linear problem. It's a complex problem. It's with exponential search space because the three dimensions are involved, but it's actually more than three dimensions. If we decompose this problem, then it becomes so many different building blocks to to stack onto each other and each building block has, you know, 5 to 10 different options to pick and choose from. So that's where in combination there are more than 100,000 options in this search space and it becomes a search problem. Again, that's not a new problem across our, you know, system research. For example, databases. Databases has query optimizer. Where a data engineer or analyst write a SQL query. And, but the execution, so SQL query is a semantic description of what they want to achieve. But in reality, based on whether you have an index, whether how the data is being laid out, how is data being sorted, or how data is being partitioned, there are different way to retrieve that data and process data. In the most efficient manner. And the query optimizer basically converts this query at the same— preserving the same semantic level of meaning and turn that into specific query plan customized towards the current state of database. And query optimizer basically is the brain of a database engine. And it's massive. There has been massive innovation in kind of the first decade of this Manila. I was part of the research group building query optimizers. But this, in the space we are operating, we are optimizing and customizing across quality, speed, and cost. And that is much more complicated than query optimizer, which only optimize for efficiency, right? So that's kind of what we build is to free up our customer, those app developers from carry the burden of learning how to do this 3-dimensional optimization and find the sweet spot, the best bot among all these candidates, the best suited for their requirements. So, yeah, so that's kind of one of our innovations as we build our platform, but it is directly fitting to our mission of autonomous intelligence.

Very cool, Lin. Love it. That's the end of my technical questions for today, but I suspect that we have a lot of listeners out there who, having heard what an amazing speaker and thought leader you are, with all the capital that Fireworks has raised, with the impact that they're making, the challenging problems that they're solving, I bet we have a lot of listeners that would love to work for you. Do you have any open roles, and what do you look for in people that you hire?

Absolutely. Um, the reason we raised Series C last year is to massively accelerate our growth. So we're actually hiring across the board, from all the way from the GTM side. We have a lot of openings from sales, marketing, as well as product, engineering, and in the finance, business operation across the board. I really mean it. Because we just have so much demand, we cannot manage, and we love, we love to work with very creative and high aptitude people who are willing to join us, take a big chunk of ownership, and drive a lot of impact. So yeah, if you're interested, I'd love to— please send, please contact us and send your application.

Wonderful. All right. So now we're just to my last two questions that I ask every guest. But I think I'm going to get an interesting answer based on our conversation before we started recording. I always ask my guests for a book recommendation, and I think you have something else.

Yeah. So I, I think this is an interesting time of AI and everything has changed and how we learn, how we learn has changed. In the past, I've been reading books from mostly around I love to read books around business, um, and, uh, you know, leadership. And I really like to read books about certain kind of individuals I'm very curious about. But nowadays, um, I, I listen to a lot of podcasts. I think, John, your podcast will be very impactful as well. Um, I And particularly a fundamental reason is the following, right? So the, this AI innovation is changing how, how we do business, how we create technology. All these agents that is emerging is going to change our day to day. But at the same time, fundamentally, it's also changing how we learn, how we learn what's happening. In a fast-moving world. Particularly, I think we have a lot of people, including my founding team, including my top-tier engineers, they learn a lot from a lot of posts. We were always on the cutting edge. We read a lot of paper and kind of the paper reading velocity is very high and paper generating velocity is very high. But we read a lot of best practices. From social media. And there are a lot of creative ideas, a different way of thinking, approaching problems. And we have to, like, you have to think differently to be very innovative in the AI time because a lot of fundamental assumptions has been disrupted. For example, we are rethinking our interview process because now AI agent is, coding agents are very good at generating code almost as a fresh graduate junior engineer. So, so then the question is, hey, is writing code quickly and correctly an important area to test or not, right? So we're questioning that because with the assistance of coding agent, that's no longer a problem. And it's more a problem to have the eyes and the mental framework to judge how good is the code. And have a strong way to steer a coding agent to design very well, like coming from the source of design and system architect thinking to drive the implementation. So it starts to shift kind of the focus and bottleneck of the software development. Just give you an example of recruiting in the hiring process that has changed. So I will say the whole The whole community is a book. The whole community is writing a book of AI. I think that's fascinating to me. And that's kind of, it's, I just feel like I'm very lucky to live in this time of this world to fully embrace fast velocity of changes. And so much, there's so much to learn from each other.

Cool, what an answer. Uh, yeah, certainly we've never had an answer like that before on the show, but I love it. You're, you're a really progressive thinker, and I have personally learned tons from this episode. I'm sure a lot of our listeners have as well. Lin, how can people follow you? Where, like, you know, where on social media, or how can people be getting your thoughts, uh, after this episode?

Yeah, I'm learning also to transition my interaction more towards social media. So you can follow me on @Lin Qiao, my handler on Twitter, X.com, but also you can follow my LinkedIn. I share my thoughts. I will do more in the future, but also I talk about our product and the product launches and direction we're heading towards. I love engagement. So if you have, for all the audience here, if you have thoughts, feedbacks, ideas, I would love to talk with you to kind of exchange notes and go from there.

Awesome. Thank you so much for making, for opening up your inbox to our listeners, Lin. Really appreciate it. And yeah, that's the end of the episode. Thank you so much for joining us. I can only imagine I imagine how crazy your schedule is, and so to take this time out and speak with me and share your thoughts with our audience, we really appreciate it. Thank you, Lin.

Glad to be here. Thanks, John.

Super episode today with the exceptional engineer and entrepreneur Lin Qiao. In it, she covered how over 90% of the world's data live in private enterprise systems and never make it into foundation models. Representing a massive untapped source of intelligence. How autonomous intelligence is about continuously and automatically customizing models with private enterprise data, resulting in millions of specialized models rather than one AGI to rule them all. She talked about her small-big-small approach, which means starting with a small model to iterate on data quality, moving to the largest model for best quality tuning, then distilling back down to a small small model for fast, real-time inference. And she talked about how coding agents now perform at the level of junior engineers, fundamentally changing what matters in technical interviews from writing code quickly to having the judgment to steer and evaluate AI-generated code. As always, you can get all the show notes, including the transcript for this episode, the video recording, any materials mentioned on the show, the URLs for Lin Qiao's social media profiles, as well as my own at superdatascience.com. Www.patreon.com/971. Thanks to everyone on the Super Data Science Podcast team, our podcast manager Sonia Breivik, media editor Mario Pombo, partnerships manager Natalie Jajski, researcher Serge Massis, writer Dr. Zahra Karchei, and founder Kyra Larimenko. Thanks to all of them for producing another fantastic episode for us today. For enabling that super team to create this free Super Data Science Podcast for you, we are deeply grateful to our sponsors. You can support the show by checking out our sponsors' links, which are in the show notes. And if you'd ever like to sponsor an episode, you can get the details on how to do that by making your way to joncron.com/podcast. Otherwise, help us out by sharing this episode with anyone that would like to listen to it. Review, uh, the show on your favorite podcasting app or on YouTube. Subscribe. But most importantly, just keep on tuning in. I'm so grateful to have you listening, and I hope I can continue to make episodes you love for years and years to come. Till next time, keep on rocking it out there, and I'm looking forward to enjoying another round of the Super Data Science Podcast with you very soon.