How This AI Startup Grew by 100x in Just 6 Months | Fireworks AI, Lin Qiao

This is a fast-moving technology. Everything's new. How fast you pick up, how fast you're a learner, how determined you are a problem solver, that makes a huge difference. Hi, I'm Lin Qiao. I'm CEO and co-founder of Fireworks AI. I started Fireworks AI with my co-founders late 2022. We have a long-time experience working at Meta, building AI infrastructure from ground up. We work with many companies in the industry and experience their pain going through this AI-first transition without a team like us, without the know-how, without proper hardware, without the right software and tools. Today we're processing more than 150 billion tokens per day and generating more than 1 million images per day. From the funding point of view, we have raised first round from Benchmark, $25 million. And then recently we just closed a round led by Sequoia with $52 million investment and post-money $552 as the valuation. So our valuation grew by 4 times. Throughout my career, I've been through waves of technology-enabled business transformation, waves of it. And that particular intersection of technology and business deeply interests me. And then we go through mobile-first transition. That's a huge tectonic shift. I want to join a company where they are on the forefront of this transition. That's Facebook. When I joined, Facebook just finished the transition from desktop to mobile-first. So when I initially joined Meta, I joined the data infrastructure team. And when I'm running the team, I look at the stats of data growth and the data growth is mind-blowing. It's just so fast, it doesn't make sense. I ask myself, I need to figure out what's going on here. And I did a drill down and figured out, oh, majority of data growth is driven by AI. That is clear to me. That's the future. And there's a lot of unsolved problems because it's a new emerging area. So I moved to the AI space, and that's why I started to build a team from 5 people to 300 people over the course of 5 years. So PyTorch is very widely adopted. For example, many of us use Netflix or other streaming media. The content on Netflix is heavily personalized. Many of those language recognition models are PyTorch models. PyTorch has been heavily used in robotics, self-driving cars. Almost all self-driving car companies use PyTorch, including Tesla. Me and my co-founder have spent years at Meta, and we have hundreds of people building out the AI infrastructure, supporting Meta's AI-first transition. While I also have many friends working in other big companies in the industry, they are way behind Meta, and when they go through the AI-first transition, they don't have hundreds of people, a machine learning team or AI infra team to help them. So our mission of starting Fireworks.ai is to enable new business to flourish building on top of this innovative GenAI technology without 100 people, machine learning engineering team and infrastructure team. GenAI is very big. The model is very large. It's just very slow to run these models. So not having very low latency and fast response makes that product not a pain at all. So we know this is a big pain point. Another big pain point is the generative AI model is so big and they have to run on GPU and they have to acquire a lot of GPU and GPU is so expensive. If they're lucky to get the GPU, it's— they have never seen such a big bill before. So we designed our software stack to minimize your GPU needs. To solve your problem. In that way, it significantly controls the cost, operation cost for the company. So then our customer has, will have a viable business when they scale quickly to 10x, 100x, 1000x more customers. Within the past half a year, our traffic grow by 100 times. Today we're processing more than 150 billion tokens per day and generating more than 1 million images per day. There are a few key things about startups. One is do not work on incremental work. Do not pick on incremental things. It's human nature to work on projects that we know we can deliver, but it's not for startups. Startups are pushing for 10x. 10x faster customer adoption, 10x faster infrastructure latency, 10x higher scale. Startups are only here for 10x. It's for a huge leap. The second is startups are not for slow-moving pace. It has to kind of outrun and outpace competition. There should have already been incumbents occupying this occupying the space, and the speed is advantage of startup because we don't have the burden of coordination. We don't have the burden of slow decision-making. Be able to say no is essential for move fast because over time it's, you can justify, hey, why we should add this, why we should add that. And then the same person got time slices into multiple different top priority things. So I think laser focus on prioritization and really ask hard question. Does this project really move the bottom line of our business? Can we visually see that our metrics will change? I think that have the principle to drive that conversation and be able to say no is very important. Again, The goal is not to make people happy. That's not the goal. The goal is to make a solid product decision or strategy decision and have everyone laser-focused on delivering that. That's the key point. Starting this company, we're constantly asking ourselves, are we working on the right problems? Can we deliver this Why not today? And can we move faster? I think that mentality is deeply ingrained into us, and that's also essential to, I think, to build a successful startup is kind of have a huge sense of urgency to keep asking, why not today? Why not yesterday? Why not faster? 2023 has been a year where we laser-focused on large language models with text in, text out. This year and beyond, that will not be sufficient because a lot of business tasks will emulate what's happening in real life. In real life, we communicate way beyond text. We speak, and we use visual to collect signals. So that's where we have to expand beyond large language model into models who understand images, who can generate images, who understand audio, who can generate audio. And I firmly believe the direction for the industry to move forward to is into multimodality. And today on our platform, we already serve more than 100 models. Across all these modalities, from large language model to audio models to image generation models. We have a very broad variety of modalities for our customers to pick and choose from. With that said, that's still not enough because every single model has limited knowledge. And the fundamental reason is those models are like child in the school. They learn from textbook and those gen AI models learn from training data. And training data is finite, it's not infinite. If you ask a question, it will have to give you an answer in a probabilistic way. When it goes out of its knowledge, it's going to hallucinate. So that hallucination is a big problem for application developers. And the way to address that is to let each model focus on its specialty, only answer questions when it's good at. And we are building a power routing layer. It's called function calling. So we have our proprietary knowledge to, to figure out which specialty model to route into to give the best answer. It does not just route into different models, different modality. It can also route into APIs. APIs could be search, could be weather, could be stock price, could be many other things. So then this model is able to pull together the totality of knowledge to give the best answer to, to our users. This is the form and shape of compound AI system. And Fireworks is growing into and building into our next-generation compound AI systems. Very simple. Aptitude. It's not about experience. I actually prefer aptitude over experience. I need to see the fire in the belly, super hungry, super motivated. That trumps anything else because this is a fast-moving technology. Everything's new. How fast you pick up, how fast you're a learner, how determined you are a problem solver, that makes a huge difference. In the early stage of my career, I have this deep imposter syndrome, so I think a lot. It's actually a way of self-reflection. I constantly think about what I can do better, how I can do things differently. But later on becomes a habit. It just When we take some action, we do things, I will observe, hey, is this going as I expected? And if it's different, then why is it different and how we can be more efficient and what are the small changes or big changes we can introduce to the process, to the way we do things, to be better, to move faster. So and then I share my thoughts with my team. Sometimes I will ask them, hey, what do you think? How this is going, what we can do differently? So just kind of build this muscle over time. We just think together as a team to kind of always— there's always room to improve.