The promise of State Space Models - Karan Goel (Co-founder & CEO of Cartesia)

I sort of see the product, uh, or the app versus model thing as sort of maybe a false dichotomy. Like ultimately we're trying to run some sort of business here. We want people to be able to use our models to solve problems for them. And I think, uh, as the model builder, it is part of your responsibility to help someone, uh, build the actual killer application for that. And that could mean actually going and building out a lot of product on top of your model. Because that's what's necessary to ultimately solve the use case. Or that could mean like, hey, here's the API, you go figure it out, right? And I think with model companies, you get to do a little bit of both where like you can't build everything yourself. So you certainly are an enabler for other people that want to build applications that are unique or interesting and that are meaningful for them. But at the same time, I think like we also look at, you know, what are the main places where, you know, for example, for voice, what are the main places voice APIs matter, and then we want to build as much as possible there to help customers, um, you know, succeed in using our models. So that's kind of how we view it. It's like our models are not useful unless we can sort of put them into the right form factor so that customers get value out of them. And that's basically how we approach the models versus app sort of dichotomy. It's sort of like build what's necessary and, um, and then obviously like make decisions about, um, where to go from there.

Welcome to AI in the Real World. I'm Jaya, a partner at Foundation Capital, and I'll be your host today. In this episode, I'm I'm joined by Karan Goel, co-founder and CEO of Cartesia. After completing his PhD at Stanford, Karan and his founder Albert took their research on state-space models to production with Cartesia. They believe SSMs could be a promising alternative to the transformer models that power most AI applications today. Cartesia's first product focuses on voice, but as you'll soon hear, that's just the first step of their longer-term vision. In our conversation, Karan shares the challenges of transitioning from academia to startups, his contrarian bet on new AI architectures, and his vision for building AI that processes information more like humans do. Here's Karan. So Karan, I feel like a lot of people know a lot about you and what you've done with Cartesia, but what a lot of people don't know is why you got into entrepreneurship. And I've heard the story, but for everyone else, what is that story?

Yeah. Um, I, was doing my PhD before starting Cartesia. So I grew up in India. I went to undergrad there, moved to the US for grad school, very standard story. I spent about 10 years doing research in AI, started in 2015 and 2014, 2015 in undergrad, and then sort of got excited about some of the work at the time that was happening in reinforcement learning, which has made a comeback since then. And so got super excited about working in AI and then ended up doing a PhD. During my PhD, I was working with a lot of really great folks in Chris Frey's lab at Stanford. And so Albert Gu and Tri was there doing flash attention. Albert was, started working on this stuff around new architectures for AI. And so kind of got excited about that work, worked with him for a period of like 3, 3.5, 4 years. And then we just thought it was so interesting that we should really go figure out how to build this for the world, more or less. So that was kind of how the company got started. It was more like a natural jump from the work that was already happening in academia that we were doing to, hey, let's use this to build great models and products. And then, you know, I rose to the occasion. I decided it was a good time to for my first job to be this. So, um, yeah, it's been fun.

Going from academia to startup life, it's, while it's becoming more common, it's still like a small minority of founders. What was the hardest part about that transition and kind of what are the mental models or habits from, from researching every day at Stanford that you kind of had to unlearn as a founder?

Yeah, I guess, um, well, there's good parts, which is like you are pretty used to things being painful. That's kind of the default mode. You, you just work through things day to day and like you aren't necessarily that bothered by fluctuations in outputs, right? So you can be pretty consistent. I think that's part of what you learn in research is consistency. And then I think the difference is just like the breadth of things that you have to do as founders, because you basically care about everything and everything tends to be very existential to you. So I think because of that, it is interesting to ramp up on, for example, go-to-market and spend a lot of time there, learn how to do sales and marketing and all these different functions that, you know, obviously I was previously unfamiliar with. And so that, that I think is kind of the most interesting part of it. And obviously the part where there's a lot to learn, but the parts that are like similar are just like you want to do great work and you want to do it with a great set of folks and you want to get other people excited about your work. You want to share that with the world. So. That part is like really similar. And so I think, um, feels very natural to exercise some of those muscles again, I would say.

All these people are on Twitter or whatnot, they're talking about how, how all the next companies that are hitting $100 million ARR are gonna come from like 10 people. I don't know whether you think that's true or not. What we're seeing across our portfolio is that people are still hiring engineers, people are still hiring AI researchers, and I think one thing that you do extremely, extremely well is recruiting. And I'm curious to know, one, are the next big companies actually gonna come from the, these 10-people teams? And then two, what's your secret sauce to recruiting? How have you, what are your tricks for the rest of the world who is struggling at it?

Yeah, I guess, um, on the engineering side, look, I actually think, um, you know, let's say you, you think about AI as being as good as the best, um, IOI, You know, sort of programmer. Now I happen to have a co-founder that was, uh, you know, gold medalist in the IOI, Albert. I kind of know what it's like to work with, uh, work with probably what the best AI is going to look like. Obviously part of it is like solving like hard, uh, you know, sort of mathy implementationy problems, but it's also like picking the right things to work on and, and just, you know, having great intuition about what's important. Ultimately implementation is only like a fourth of what needs to happen. I don't see that part going away because like when we recruit, for example, we're not recruiting for, you know, people who can just write code, right? That's like a basic skill that great engineers will have. It's more like how well can they actually help us in our larger goals, right? And so I don't think that goes away. I do think like you will see companies that are, and you're already seeing 10, 20, 50 person companies like that are, uh, you know, getting to $100 million revenue. But then also I think it's like, what kind of problems do they choose to solve? What kind of problems do you think are important? Do you actually need more people to solve the set of problems that you think are more interesting or important in the long run? I think that's kind of part of it. So, you know, I don't really think much about that actually, more or less I'm just like, hey, we're working on the set of things that we think are interesting and this is, if this is how many people it's going to take, it's how many people it's going to take, you know what I mean? So, um, that's part of it. On the recruiting side, I think like, Yeah, I would say we try to like recruit for folks that we think will be around for the long run. And I think part of my recruiting strategy has been like bifurcating in two directions there. One is I think I really like hiring fresh out of school talent because I think that they actually are just starting out in their careers. They're really sharp and they often, uh, are really excited to be in, in it for the long haul. With a set of people they really like working with because they're just sort of like, hey, I just want to be around people that I love, uh, hanging out with and, and I want to build cool stuff, right? Like that's kind of the vibe that they have. And so I really like that because it's really aligned, I think, with, with how I think about building teams, which is like, you should really find people that want to be around for the next 10 years, right? You can invest in them for the long term. They also come in with a little bit less cynical about like industry and things like that, which I think is nice, right? Because also I've never had a job before. So I don't have any of that cynicism and I would not want to, you know, necessarily have a lot of folks that have like cynicism around it. And then there's the like more senior side, which is like, obviously we need folks who've kind of been there, done that. And there, you know, a lot of the recruiting side is really just going in and trying to like pitch to them, you know, what the vision for the company is and really kind of, I think showing to them that we really care both about research, but we also care about commercialization because a lot of senior folks are like, Hey, like, I get it. I've seen so many companies before. They all have some story around how they're going to change the world. But like, tell me what the plan is, right? Like, how are we going to get there? Right? And that's where the maturity part comes in. So I think like, it's just really good reps to go try to sell to those folks that this is the place where they should spend the next, you know, 5, 10 years because they frankly like been at a few places. They kind of understand the lay of the land. They can go wherever they want. So really like it is about like aligning them. To your, you know, sort of vision for the world. So I think like that's kind of the two modes I would say that I spend doing recruiting. It's like, you know, younger folks, getting them excited and they just want to be around people like them. And then folks that are just like looking for a challenge and can believe in kind of the trajectory.

And speaking of vision for the company, what is the vision for the company? Like what does Doofy do?

I think to understand where we want to go, it's helpful to know where we've been. We spent about 4 years in academia doing research on model architectures. Now, the background on this is, you know, in 2016, the transformer architecture came out circa 2016, and that was really a way for folks to start building models that could, you know, really take advantage of compute and data in an unbelievable way. And so that obviously led to this whole revolution in how we build foundation models, basically, right? Like we pre-train models on massive amounts of data. They learn to have fairly general capabilities, and then you can sort of post-train them to get, you know, high-quality performance on specific tasks. That was kind of the paradigm. And so in academia, you know, Albert, who's my co-founder, he really started thinking about like, well, where does this break in the long run, right? Because if you're in academia, you don't care about problems that are, you know, happening right now. You care about problems that will happen 10 years later, right? And so his point was like, look, the way that these models are scaling, the quadratic scaling of transformers, is going to be a huge bottleneck when you start trying to put a lot of information into them, and you really care about them from the lens of like being a human-like reasoner, right? Which is something that can react to an environment very, very quickly, that has low latency, that runs on a, you know, relatively small amount of power, that can have like a memory and interact in that way. So very similar to like that human kind of like intelligence piece. I think that was something that he was like, hey, like, um, you know, quadratic scaling is going to be a huge bottleneck for this. Both on training and on inference. So we need to kind of think about this problem from first principles again. That led to all this work that we then did on state-space models, which, you know, he, he really pioneered. I was kind of helping build a lot of the, uh, applications and the, you know, rubber meets the road kind of side of things for SSMs. It's just like, how do we understand how this works on like audio data? How do we understand how this helps us with video understanding, with like image classification, with dealing with EKG data, with EEG data, with like sensor data of all kinds, right? Like that was kind of where we were spending a lot of time, but really it was that, you know, architecture piece that was very interesting. And so I think we made a lot of progress in academia there in building these new sub-quadratic architectures, Mamba being, you know, one of the flagship, let's say, architectures now. And so from a company perspective, I think the mission was very much the same, which is How do we now build a really powerful multimodal model that can take advantage of all this new technology and can be deployed on a very small amount of compute in order to interact repeatedly with people and environments? That's kind of what we want to do. And so within that, it's sort of like, hey, how do you get this stuff running on device really, really efficiently? So you can basically power this with small accelerators. How do you handle multimodal data natively? You know, how do you basically have models that can learn from the raw information? A lot of what you see in machine learning has been through the lens of text. You know, text is sort of like a human abstraction on the world, right? And I think part of the thing that we think SSMs could be great for is remove the human part of the abstraction and really learn from the raw data. So build the abstraction in the model itself. And this is kind of the more like deep learning, pure, you know, sort of approach, right? Which is like no feature engineering. And so if you think of text as some form of feature engineering too, it's like, oh, how do you get rid of that and learn from the raw data directly? So that's kind of really interesting. But our, our real, real goal is like, how do we enable this sort of human-like intelligence, right? Like models that run on device that are really fast, that you can repeatedly interact with, that remember you, and then you can sort of use that to, uh, you know, do tasks for you, act on your behalf, all of that. That's really where we want to go. And, you know, we've broken that problem down. So we've decided, hey, like, we think the first place to build this is for voice because voice is a really powerful sort of medium of communication for us as humans. It's also like one of the ways that businesses interact with customers. It's the way people create content and disseminate that to other folks. So there's a lot of different applications of this in voice. And so we were like, hey, let's build this for voice first and then go expand to other modalities and build that whole vision. next. And so that's what we've been doing. We've been really focused on kind of doing this, but in a smaller way for, for voice first. Um, yeah, that's maybe the, the very, very, very long summary on, uh, on all of the history of, of what we've been up to.

You mentioned the subquadratic architecture and, and SSMs. If there's non-technical listeners listening, like, how do you describe to them the sort of differences between the two architectures and like, what are the pros and cons of each?

A couple of salient things that I think I would focus on when trying to, you know, understand the differences. So one is, um, you know, I think of transformers as basically retrieval machines. You take lots of information, you put it into something called a context window, the model reasons over that context window and then produces outputs that match what you care about. So you can take your documents, put them into the context, ask a question, and you'll get a response. So they really function as almost like these, uh, pull information into context, ask a question and get an answer type of a machine. Um, I think in the SSM case, it's actually more imagining, uh, what a model would do if it looked like a human. So with humans, we actually, um, are streaming information into our, our, our brains because we're, we're reading, we're interacting, we're receiving audio, receiving video. So it's, it's very much a streaming system, right? Rather than this sort of like retrieve information and bat system. And, uh, and so because of that, you know, the way that these models actually understand information is different. They can't access all the information at once. They actually stream the information through and then they try to memorize what they think are important facts and then use that to act. So because of this, it's almost like there's a memory that the model has. It like humans have memory. It's like the model has a memory. And so the model is constantly updating itself in response to new information and then using that to act. Whereas in the transformer case, it's more that you're pulling information in as you need it and then answering questions against it. That's kind of maybe the one of the more salient differences from a little bit more non-technical hand baby point of view. And then, um, I think one of the fundamental differences from a more technical perspective is just the role of compression and memory inside these models. SSMs view that as sort of a fundamental primitive. So it's sort of like, it's important to take a lot, as you take lots of information in, it's important to compress it into a memory so that you can remember lots of things and stuff them into that memory. It's sort of like, uh, you know, analogous to having like a bunch of files on your computer that are, you know, available raw versus like putting it into a zipped folder and then compressing it, right? So the SSM is the zip file version of it. And then the transformer is sort of like saying, I'm going to keep this in its raw format. And because of that, you get the efficiency differences too, because like transformers will have that information represented exactly while SSMs will compress it. So that's why SSMs tend to be faster, more efficient, et cetera. So that's kind of the, the big thematic differences between these models.

I love it. What are the advantages that, you know, SSMs and the architecture that you guys are using offer over the transformers in real-world applications, and you guys already have tons of customers today. What are the particular use cases where they're especially compelling?

Yeah, um, so with SSMs, there's a couple of things that are really, uh, great out of the box. So one is, um, they're very efficient. So if you need to scale, uh, throughput, uh, inference efficiency, you need to be able to, uh, do tons of requests on models, that's where they shine because you get like really great efficiency out of these models. and, um, one example of a use case where that matters is, for example, uh, you're making voice calls, right? There are, you know, in parallel, so many voice calls happening in contact centers, for example, where you're really kind of calling out to models in parallel at really big scale. And then, um, you essentially want to be able to spin up these almost like, uh, human-like voice agents and then have them talk to customers and, you know, do phone calls for, you know, inbound, outbound. Whatever the use case might be. So that's a place where having this type of architecture and using it to build these models is actually very advantageous because it allows us to kind of have much more scalable infrastructure for customers. And so they actually see a direct impact. Now, customers don't come to us and say, like, I want to know what architecture you run underneath the hood, because they don't care. But they do care about latency. They do care about, like, how much you can scale for their, like, use case. They do care about reliability. And all of those are places where actually you see direct impacts of the work that we do on the architecture side. So that's, you know, one place where we're seeing that have actual direct customer impact. The other is we think that, again, from an on-device perspective, when you're actually fundamentally constrained on compute, then it's not possible for you to just build another data center and then, you know, you know, essentially exchange compute for inefficiency. You actually have to think deeply about how to make the models fast within the constraint that's available to you. And so on the on-device side, we really see kind of a huge opportunity for these models to get deployed. And we've been doing a bunch of work there as well, uh, that, you know, again, we are sort of, uh, maybe haven't talked as much about publicly and we'll hopefully, you know, have more to say on, um, in the coming months. But I would say that's definitely an interesting, interesting area where, you know, you can imagine the entire stack needs to be re-engineered to bring cloud quality into sort of mobile footprint, right? Like, that's kind of how I think about it. Like, how do I get like What's running in a giant data center running on your phone with that same quality. And so I think that's where like these new architectures can actually be a bridge to give you that like secure, private, instant kind of communication with these models, but then have it run on, on something that's like the size of your phone. So yeah, those are a couple of places where we really see kind of big lifts. And then there's more on the research side that we're continuing to kind of push on, which we think again will be, will be interesting and offer advantages on quality and like you know, hopefully breakthroughs and also what you can learn from raw data. I think that's another big theme here.

You have the choice to either be a model company or be an application company, and I'm sure at times you probably thought about either or, and obviously OpenAI has its products like ChatGPT and, um, their APIs and whatnot. How do you think about your product and how you're going to commercialize? Do you think that there's risk? What are the pros and cons of both approaches?

You know, I sort of see the product, uh, or the app versus model thing as sort of maybe a false dichotomy. Like ultimately we're trying to run some sort of business here. We want people to be able to use our models to solve problems for them. And I think, uh, as the model builder, it is part of your responsibility to help someone, uh, build the actual killer application for that. And that could mean actually going and building out a lot of product on top of your model. Because that's what's necessary to ultimately solve the use case. Or that could mean like, hey, here's the API, you go figure it out, right? And I think with model companies, you get to do a little bit of both where like you can't build everything yourself. So you certainly are an enabler for other people that want to build applications that are unique or interesting and that are meaningful for them. But at the same time, I think like we also look at, you know, what are the main places where, you know, for example, for voice, what are the main places voice APIs matter, and then we want to build as much as possible there to help customers, um, you know, succeed in using our models. So that's kind of how we view it. It's like our models are not useful unless we can sort of put them into the right form factor so that customers get value out of them. And that's basically how we approach the models versus app sort of dichotomy. It's sort of like build what's necessary and, um, and then obviously like make decisions about, um, where to go from there.

That's— I would love the take. What are some of the best practices for building voice AI products that you guys have learned? Because You guys have clearly done some complex, uh, you know, customer implementations including Quora. What are the learnings there from going beyond the model and actually building some sort of application layer tooling for them?

Yeah, I think, um, you know, on the voice side, there's a lot of complexity right now in actually getting to, uh, you know, voice agents that work. Let's take the, you know, voice agent side of this and the voice calling side. I would say Uh, tons of companies are trying to understand how do I deploy voice AI to automate some form of phone calling and maybe even to expand where I can take calls of where I previously could not. I think right now we're in this phase of experimentation where a lot of them are looking to understand what is the best way to implement that as a solution into their enterprise, let's say, or even, you know, a mid-market segment. And I think there they're looking for, um, obviously from the, from the model provider side, I think they, we often get a lot of questions about like, hey, what are you seeing? Because you're, you're sitting at the bottom of the stack. You probably talk to everyone. Like, what are you seeing about how this is getting implemented? And I think, um, main themes there are a, like, a lot of it is about the workflow, right? So there's a lot of, you can't just build generic orchestration. and then put that out there, you have to help them sort of understand how to like program those workflows too. I think because ultimately like the end value comes from feeling like the entire system succeeded rather than like, you know, the voice succeeded. And so they're like, obviously we, we work with a lot of the, um, companies that are, uh, doing either like, um, end user applications of specific types. So they're working, let's say in healthcare or they're working in specific industries to help them. Do voice calling in specific use cases. That's obviously one part of it. And we're seeing a lot of fragmentation there, but also it's because it's necessary, right? Like there's a specific use case and somebody has to go build a workflow for that use case. So that's what's happening there. And I think on the, on the actual, like, how good are the, you know, voice calling agents? It's super natural now. We, you know, power a lot of voice agents with Cartesia models. They sound amazing. It's almost indistinguishable from humans. Actually, the only place you can distinguish it is typically like you're having really long responses from agents, and then you can start to see small differences because humans are very good at picking up little, little things. But like, you know, you can get pretty indistinguishable quality, which is suggestive because like, if you can, um, you know, if folks are really excited and okay with talking to, you know, an AI on the phone, Question is like, what are the, uh, obviously the easier problems get solved, but what are the hardest ones we can now start to solve? And that's what folks are also thinking a little bit about while they're implementing a lot of the easy stuff, which is like, you know, hey, you're going to call me for a refund. Obviously that's an easy phone call for me to throw to an AI because you're incentivized to stay on the phone, right? Like that's kind of the type of phone call that's easy to automate early on. But then like, I think folks are also thinking about, hey, like this seems like it's really going to work. So where else can we start deploying and how do we prepare for that? And so that's where I think there's like interesting stuff happening, which is like, I think enterprises are trying to figure out like, should I go, you know, build like and buy like 5 different point solutions for my 5 use cases? And how do I prep my internal infrastructure to be able to support these? Because this might be a big thing and I need to be ready for that. And so that's a place where I think there's a conversation happening, which is like, how do we build this out so that like, if we have 20 use cases, it's not going to be hard to do those 20 use cases. I don't think there's great answers yet there, but I do think that's like a pretty interesting kind of, uh, uh, theme that I've been seeing at least more recently.

Yeah, I, I think it's like the notion that AI is like— AI is actually a pretty consultative sale, uh, what we've seen across the map. Maybe switching gears a little bit to the broader AI landscape, I think some argue that AI breakthroughs will come through from better post-training techniques, fine-tuning, RL, you know, what you've studied, retrieval, reasoning, rather than like a new architecture. What is your take on that? Do you agree? And if so, are we still in the early days of architectural innovation, or where are we?

Yeah, um, I think that, you know, there's many axes of improvement, um, in AI. I think it's good and healthy that people disagree on what the most important thing to work on is because, you know, we'll make more progress collectively. On the architecture side, I do feel like we're in sort of the, the early part of the game there because there is so much to, uh, to be done in terms of actually, um, building like bigger models, better models, actually deploying this at scale. And that's kind of where we come in, at least on the architecture side. I think similarly, like, there's obviously a ton of work to do in in multimodal training, pre-training actually. I think there's a lot of interesting problems in multimodal. For example, like I would say, we don't really know how to train on massive amounts of long-form video data. There's not really very effective ways to do that. We don't really have great world models yet. Like they're brittle, like obviously the early versions for VideoGen are like interesting, but like they're not fundamentally like simulators of the world yet. And so that's clearly a big axis of improvement for AI that you can imagine being very important for things like robotics and planning and stuff like that. So that's clearly like multimodal pre-training on large-scale data is going to be something that's quite interesting. Then there's post-training and generally the question of like, how much data do you actually need to train great models? I think that's like, how big do models need to be? How much data do you really need? And what's the intersection of quality versus volume there? I think these are all questions that, you know, keep getting relitigated. Um, and like, there's still a lot to figure out there is my feeling. So that's also pretty interesting. I think the reasoning stuff that's, uh, you know, been kicking off is super interesting. Like, you know, how do you get models to think quickly and deeply about problems? And I think right now, obviously a lot of it is in code, which I think is like interesting, but you know, that's fine. It's, it's kind of like sandboxy, you know, to me, code. Like I think connecting it to more like analog stuff is, is where I think it's more interesting and hard. So then there's a, there's a whole area of like, how do reward functions and models work there? How do we like actually basically incentivize these models towards the right things that we care about? That's a whole area that I think is ripe for more disruption, interesting breakthroughs. So I just think like the, basically the field is wide open actually. We kind of got to like a pretty good first state with like the language model revolution. But I think actually if you're an AI researcher and you care about the field and you want to build great products, like there's a lot to do, right? Like there's, it's, it's like limitless the number of problems that are out there. So I do feel like that gets me pretty excited actually. Architecture stuff is obviously one thing that we care about personally, but we also think about like broadly what are the bigger problems in AI and like, you know, where, where to spend time there.

In terms of, you know, bigger problems in AI, I feel like the narrative between open source and closed has become a very big sort of battle, especially with, you know, Deepseek, and I would love to get your thoughts on that as well. But what do you see as like, do you see the open way AI as a serious alternative to the frontier models like OpenAI, or is the gap just really too wide?

Here's what I'll say. You know, I got this question in all hands 2 weeks ago from someone at the company. And, um, I think the, the sort of thinking behind the question is like, how does one, even as a company, think about open source? And, you know, what is the role of open source? I would say like, one, one, I would say OpenWeights, not really open source because the process of building the model is still completely obscure, right? So, so it's not really true open source at all, right? Ultimately, if you're building products. For people, somebody has to care deeply about obsessively grinding out and improving every detail of the model and the product. And I think with open weights models, that's not something that's incentivized. So people throw out a lot of models, but it's very difficult to imagine those models then becoming great products unless somebody else says, I'm going to obsessively make this better for one use case or one set of customers, and it's going to be the best there. That's where the companies come in and the, the non-open source or OpenWeights part of AI comes in, where like you actually have teams of people that can, A, access the training procedure, the data for building the model. They have the model itself and they have the customer and the product, and they iterate deeply because they, they have somebody at the other end saying, I need this to be better in this specific way. So I think as long as that feedback loop is missing, which it is in the OpenWeights case, I, I can't imagine there being a catch-up. Because ultimately if you're a customer or a user, you want the best thing that works for you, and somebody has to go deeply, uh, deeply kind of invest in that. And I think in open source software though, which is the difference between OpenWeights AI and open source software, there's a team of people that obsess about that product and that piece of software, which is why we have amazing, you know, uh, you know, software that we use every day, uh, that, you know, is, is widely disseminated, right? Because there's obsession there. There's people that really care about it and they want to make it the greatest thing ever. I think with open weights, it's just a way for companies to throw out weights and tell people that they're doing something, but actually like you're not really helping someone build a great product, right? And I think that's the difference. So we don't even try to really do that much work in the open source, open weight side because we think like, what's the point, right? Like we don't want to be playing like a, you know, you know, sort of almost like a two-sided game there. I think we'd rather invest all that time in trying to understand how do we make our own models better for customers, and then how do we drive down, you know, pricing and all of those things so that it's actually beneficial to users. So that's kind of my perspective on it. I think it's sort of like interesting, but I haven't really seen like it stand the test of time, right? Like we've been talking about this for 5 years. There's some models that are certainly useful and like folks have put time and, and invest effort there. I'm not saying it's like universal, but I think this sort of maybe the broad strokes themes I would say that, uh, that stand out to me at least.

Yeah. How, how would you advise founders thinking about your models and even OpenAI's models, uh, versus thinking about building applications on top of them? How should they think about the competitive risk of like, okay, let's just say one of the foundation model providers is getting into the application side of things and they're building an application on top. How should they think about that dichotomy, especially if they don't want to train the model and don't have the expertise to do it?

I think it's good for folks that are excited about applications to focus on those applications and build them to the best of their ability and, you know, build the best version. And I think they'll do a much better job of it than the model companies, unless the model companies deeply care about that problem, right? And the example here is like Google built Gemini. Gemini is a great model. but they're now putting into every product that they own, right? And it's not going to be a rework of the product from the ground up. It's like, well, we already built Docs. We already built Gmail. We already built, you know, YouTube. We already built this. How do we stick it into that as quickly as possible so we can start distributing it, right? And I think somebody starting an application company is going to have a very different perspective on it because they're not tied to something they've built before. They are excited about the potential of this. They think the workflow can change actually, typically. And so they actually design it quite differently because of that. So I do think like there's a lot of value actually. And it comes a lot from like trying to reimagine how the problem is solved because now you have a different set of tools. So basically I think they should view AI as a set of tools and then they should really focus on those customers. And I, to your point earlier on, um, there is going to be huge companies that are built because they really kind of obsessively built a, a new type of product for a specific thing that matters a lot. Uh, coding obviously is one of them.

Perfect. Perfect. All right, now we're gonna go to the, you know, sort of the, the fun questions, I guess. And this is what makes the podcast go viral. Um, uh, so you're, let's just say you're Sam Altman, Deepseek just came out. What are you gonna do? Do you open source your models? What's your strategic decision?

Um, nothing. I would do absolutely nothing. I would just be like, yeah, it sounds great, let's move on.

That's good. Good take.

Yeah.

Okay, cool. Um, what's a fundamental assumption about AI today that you think will be proven completely wrong in the next decade?

I think that, you know, Speaking to the work that we're doing, that, um, the transformer architecture is enough to solve every problem in AI. I think that there are a lot of problems that people are underestimating how hard it will be to make progress on, and new architectures are going to be necessary there.

If you had to short OpenAI or Anthropic, which one would you short?

Um, I would say Anthropic. I think OpenAI is definitely like Um, yeah, I'm a believer to some degree in the way that they're doing their stuff. Yeah.

Interesting.

And I think part of the stuff is like multimodal, et cetera. I think they have a pretty interesting roadmap, at least that they've shared publicly. I think Anthropic shared some of their roadmap too. Um, and maybe, you know, I'm not, I don't fully understand it in the way that I do with the OpenAI one. And so, yeah.

Yeah. Which incumbent CEO is navigating the world of AI, like, correctly? Which one do you admire the way they're navigating it? And then which one do you think could probably use some advice?

I'll give you a few that I think are doing a really interesting job. I think on the app side, I would say Arvind from Perplexity. I think he does an amazing job in, uh, in sort of understanding what his role is as the founder of that company and just, you know, sort of obsessively pushing that forward, at least from the outside in. Where I think maybe less so, I think like, I don't know what, you know, some of the other like AMDs of the world, Amazons of the world are up to really. I think it's a little confusing to me and maybe it doesn't matter as much for them, but you know, the stories that they tell around AI is a little confusing to me, to be honest. Like, I think they could be doing a lot more, actually. Same for Apple. I just feel like it's such a miss that these companies, like, have, you know, they produce, uh, HomePods and they build Alexas and, and all that, but they don't really have great, uh, great models, great products there. Um, just feels weird and laggy.

Yeah, that's fair. If you had unlimited compute and funding, which experiment or AI breakthrough would you pursue?

I think that it would be quite interesting to try to train models on very, very, very long context data and like try to design datasets that are just truly like trillion token length and train on them just because I think like scaling in the sequence length direction has not been a big theme. It's, you know, obviously there's like scaling and compute and data and there's a test time scaling stuff, but I think like just how much information the model sees and if you can carefully design that almost like show the model a lifetime and have it learn, right? It's sort of like interesting to me. That would be an interesting thing to do with unlimited compute and funding. Oh, for sure.

Interesting. If Cartesia is IPOing in 2028, what are they IPOing for?

Um, if you ask me right now, you know, certainly if we're IPOing, we're, uh, you know, succeeding commercially right now. The biggest opportunities I'm seeing are obviously operating in audio are in like, uh, contact centers and creation. So I think it's really like nailing those, um, and, you know, scaling them globally.

Maybe from here we jump into like the what's next for Cartesia part. Like, I think you've tweeted before, the audio is just the start. So what is next for you and what new product directions are you most excited to explore beyond Sonic?

We're pretty interested in the end-to-end interaction that you have with the model. Um, obviously for us, the first product we released was synthesis, which is creating audio from text or from scratch. With end-to-end interaction, I think the reason it's interesting is because there's a bit of, hey, I need to understand where you come from, I need to respond to you in the way that's most appropriate to you from an empathy and localized perspective. And then there's the intelligence piece. So I think that's kind of what we're really obsessed with is like, how do you build that end-to-end system and then just make it great on device? That's what we want to really want to focus on. So all of the work we're doing is in service of like building this, like, you know, what I call a robot brain and, and then putting it on device. That, that is the next, you know, set of things that we're working on. And, and there's lots of work to do there, I would suspect, over the next few years. So if you're just trying to wrap our heads around it, put out stuff that we think people will like and want to use in the meantime and keep chipping away.

What assumption do you think you're making that you have the least confidence on when you think about the next 5 years for Cartesia?

Yeah, I think that actually the, the thing that I have the highest belief in is also the thing I have the least confidence on, which is, you know, architectures matter is something we really truly believe, but it's also something that's an unknowable, right? And so, you know, from a probabilistic point of view, Who knows, right? Like what the future looks like. Lots of people try to predict what's going to happen in AI. Most of them are wrong all of the time. So, you know, I, I think that's where I would say I place the most risk, but that's also why this company matters. It's like, you know, there is a clear reason that we exist, which is to understand and answer this question. If we answer this question in the affirmative, I think there's a, a massive opportunity there. And I believe that even if we answer it in the, in the negative, we'll still actually build something great along the way. So that's kind of the, like, you know, shoot for something that's really hard that you believe in, that you don't quite understand necessarily all the way, and then, you know, you'll end up somewhere in a pretty good place, I would say. That's kind of what we at least believe.

I think what's actually really cool about what you do is you are— you've taken a very contrarian approach to what the rest of the world kind of already believes in this transformer architecture, and you've convinced so many people to join you. You've convinced customers to use your product today. Like you've convinced investors to invest in you and to believe in this new architecture. What advice do you have for founders that are raising, you know, and what they might have is just a research paper and 4 years of hard work and belief in like what they've built. What is your advice for them?

Yeah, I think like, um, especially for folks that are transitioning from research into, um, founding companies, I would say that I think a lot of it is about like, can you find your crew before you start? I think that's really important. You have to have, uh, people you can trust and work with that can, you know, help build the company because ultimately, like, I think the, the difference will be that like, you need to be able to lean on them to help you in so many parts of the business, which you will not understand before you start. So that's so important. It's so hard to build companies. Without teams that are cohesive and so on. I mean, I, this is my first company. I don't really understand what it's like to do it in any other way, but you know, I would say I, I would find it impossible if I didn't have the support of all the folks that I work with because it's just like so important. So I think that's really important. I think like the talent and people piece of this is actually, I would say, outsized, like 80% of the, the, the real like part of company building is at least my perspective. And on the research side, I think it's like having an authentically good story that like you believe in, right? It's really easy to suss out when somebody's, uh, you know, sort of telling you a story that they want you to believe, but they don't actually believe themselves. At some level, people are very good at pulling that apart. And, um, you can look at signals, you can look at, you know, how you've spent your time, you can look at like you know, what your motivations are. And I think if you actually do care about the research, then you should do it. Otherwise, I think it's not— I personally believe academics shouldn't waste their time building companies on random research projects unless they actually think that research project is going anywhere. I know a lot of academics start companies so that they can sort of build a team and then get a nice cozy home in a larger company, which is fine. Uh, but like, you know, and, and there's some game there with, with VCs that get played, which, you know, I don't fully, you know, claim to understand. But, um, but I think like, you know, ultimately I'm, I'm operating under the assumption you actually want to do the work that you're doing. And under that assumption, I would say you would want to find the best, you know, team that you can work with. You would actually want to have the, the right belief in the work that you're doing. So I think if you have those things, I suspect somebody's going to be willing to take a bet, uh, because they'll kind of see the glint in your eye and they'll be like, yeah, like, you know, maybe there's something here. Who knows? I mean, the point is VCs are supposed to take bets, so It's as good a bet as any, right? Yeah.

And speaking of VCs, how did you, you know, what advice do you have for founders that are going through the fundraising process and picking their VC? How you, you had many options at the table. How do you pick?

Um, yeah, I mean, it's tricky. I would say it's mostly about like, um, I think it's honestly like vibes, um, partly, honestly. I mean, like, You know, at the end of the day, like, it's— these, like, funding processes run so quickly that, you know, it's not like, you know, when I started my PhD, you know, at Stanford, we do rotation programs. Uh, we rotate and spend a quarter with each professor that we might want to work with. And you have like 10 weeks to work with them to figure out if it's a good fit, if you're going to be able to work together for the long run, you know, et cetera, et cetera. That does not seem to be the theme here. Like, you're making decisions after, like, probably spending a total of 5-7 hours with someone. So it's a lot. And unlike, you know, employees, you're not going to be able to easily fire your VC, right? Like, that's not a, that's not something that typically happens. So it's tough. I would say that's like the part of the process that I honestly couldn't really wrap my head around. I think I lean on just like intuition there and like, who do I like and want to work with? And that's basically. You know, all there is to it, I think. I don't know if other founders— I think second-time founders are probably a lot wiser, but, uh, you know, they have lots to share there, I suspect. But yeah.

Do you think there's too many researchers being funded these days, or do you think that researchers in general— I know you can't generalize, but have the right motivations to start companies?

I think there are too many researchers being funded today. Yes. Yeah, I do think so. I think it's like hard to imagine this many companies doing model work will emerge at the other side of the tunnel. So we'll have to see. But I do think like, if you're a researcher, you know, building a company, I think a lot of them get it right because they care about the customers.

Yeah.

Because at the end of the day, businesses exist to serve some market, uh, at some time scale, right? You have to— somebody has to care about your product or model at some time scale. You can say, I'm going to moonshot this and spend 10 years in the lab. That's okay. But after 10 years, you're still going to be accountable to some customer. So I think that's the, I think a lot of researchers maybe don't understand that. And I think I'm not claiming that I had some special insight here. I think like I happened to be lucky in the sense that I worked with Chris, uh, Chris Ray, who, you know, has started many companies. And so he kind of drilled that into me basically on day minus one. And, and it sort of helps, I think. Because you kind of have a different perspective on what this whole thing is about.

Yeah, that's a fair take. What are people building too much of? What do, what do you see too much of, uh, that people are building? You're like, man, I just want to tell people, stop building this thing.

I think it's good that so many people are actually trying to build companies because I think that's the way you get to like some truth on like what matters in the, you know, the sort of AI product landscape. What actually is the killer applications in terms of like what people want to use and what should actually get built. So I think that exploration is actually really good. Um, and, and then there'll be a natural phase of like some consolidation, right? Uh, as with everything. Uh, but I think that's like the natural order of things.

Yeah. I mean, this was super fun. I guess last question is like, you know, what do you think, what are you most excited about in, in AI or maybe the world or any tech trend that like really you're like, man, this keeps me going every single day.

Especially given how much time we spend in voice right now, I think the fact that like there's so much stuff going on in voice and like people are genuinely excited to use products there is actually quite energizing, I would say. Because when we started the company, we decided to, you know, focus on audio. And at that time, actually, uh, you know, year and a half-ish ago, people weren't, you know, it wasn't in that takeoff stage of voice. And so it was kind of strange that we were working on this thing that may or may not have impact, but I think since then, like, actually a lot more has been taking off in the voice side. So it's pretty exciting to be part of that. I think that's kind of what I would say is kind of most interesting right now for me.