Voice AI Founders chat: Vapi X Cartesia | Agentic AI trends, Founder journey, Biggest bets in 2025

Appreciate you guys for coming today. This is our first event in the new office. So, woo!

Woo!

Woo!

Woo! Woo!

Good, good. I'm Sharad. I'll be your MC for the evening. How's everyone feeling? Alright, I have a question for you. How many of you know what WAPI means? No, I'm gonna cut off the gang.

Anybody new?

There you go.

And half of your heads just exploded.

Alright, I have a couple of questions for both of them and then I have individual questions. And we have a rapid-fire round at the end. Keep it simple. First question, why is AI such a hot topic right now? And we see many startups focusing on building agents, and many are saying 2025 is the year for agents, these voice and everything. So how real is the hype, and what do you think?

So, um, that's a good one. You want to take that one?

Okay.

How real is the hype? There was definitely a hype wave from like March December of last year, like that's when we saw our growth initially. I think a good like 15 or 20% of that YC batch was like all voice companies. And it's because all of a sudden the models had just kind of reached human performance and all these startups went out to try to like deploy these models across like various verticals. And so there was a ton of excitement around the same time you had ChatGPT Voice, come out, and when that dropped, everyone also got super excited and started to inspire lots of other startups and stuff like that. And kind of showed enterprise that, oh, shit, like this is actually kind of possible. And so, we saw that huge, huge, like, hype cycle happen, and then all of a sudden, I think there's been like a recent, like, slowdown where the market is kind of like looking around and being like, okay, this actually has to work in production. And it actually— like, it actually has to, like, pick up someone's name, correctly and actually not tell someone that they have a disease without verifying their identity. And so like those real-world kind of challenges is what we're starting to see everyone run up against. And I think that reality check has made things slow down a bit. But of course, I would expect 2025 to come back once a lot of those are solved. Yeah.

[Speaker:KARAN GOEL] I think that's a pretty fair take. I think last year when we— Initially we were doing our seed round, which was in Q1. I think a lot of folks we were talking to about audio were not very excited. This is on the venture side. Because they were like, well, audio is kind of worse than video from an interestingness perspective. And I think that kind of changed actually, because almost immediately after we did our seed round, it actually exploded, right? And there's so many use cases. We kind of are in the model side. So we build models for generating synthetic speech. And we're sort of slowly expanding the portfolio of models that we provide to basically any kind of audio capability. That's kind of the goal for us this year. And so I think what we're excited about is more adoption on the agent side because that kind of fuels more growth on the model. Layer because there's more of a need to solve kind of use cases. And on the model side, it's really about like helping solve the last mile challenges because I think the history of AI is sort of like engineering work happens in order to compensate for last mile issues. That's, for example, what post-training is basically. It's like the pre-trained model is not good enough, so we have to do post-training in order to fix it. And that's basically a bunch of engineering. And so our role is basically to figure out how we can remove these things that are brittle in the model. And I think hopefully you get to learn a lot this year about what those actual problems are.

Yeah, and I remember what those problems were too. Like, I remember in the early days, we had like— early days was like 6 months ago. But like 6 months ago, like the models would scream like sometimes without like any reason, like at people. It would just be like, oh, how's your day going?

Ah!

And it would like completely go off the rails and then we'd have to be like, oh shit, sorry, it's models, man. But yeah, recently across the whole stack, I think the reliability has just been going way up, not just uptime, but also determinism of the models.

Yeah, yeah, yeah, totally. Yeah. I think it's going to be a big theme to just get really high-quality, reliable, controllable models. You can sort of just close your eyes and run the model, and it's going to do exactly what you want it to do.

Yeah. And then touching on the this year. I think last year we had a bet that Apple Intelligence would kind of drop and then kick off this, like, crazy hype wave in voice, and that would kind of, like, drive mass consumer adoption of voice. But they were way later to the game than we thought. And OpenAI, like, has released, like, the Realtime API and, like, ChatGPT Voice. But again, like, the distribution of this tech amongst, like, people, like, the rest of the population is still quite limited. And so I think it's going to take an Apple or a Google to then get these things on everyone's devices to the point where they're actually like talking like people and you start interfacing with things more like they're people than like tapping and typing. That will be like the real inflection point that will then kick off, I think, unlocking the potential energy of the tech and where it's at today.

I agree with that, but I'll disagree that Apple We'll do it. I'm not bullish.

That's fair. Yeah, this is his job.

Maybe we'll do it.

Oh, yeah, you guys are doing on-device.

Yeah, we are actually. We're doing a bunch of work on on-device. And I think for end-to-end voice systems, that's a pretty natural place to put those models. I think it's pretty challenging actually to put these cascaded approaches on device because that's very complex. And so part of the challenge is how do you build an end-to-end model and then optimize it to run quickly on your device, and then you can have secure private conversations, which is what I would imagine is very aligned with what Apple would want to do, but feels like they're a little behind. And on the ChatGPT side, I think voice mode is not So I was actually using VoiceMod this morning because I was brushing my teeth and I had to ask a question and I couldn't type because of that. But it's not very well connected into your device, so you can't really do much with it is the thing that I don't like. But yeah, that should be easily fixable. I also saw Perplexity put something out today.

Yeah, they did like an agent.

Yeah, I haven't used it yet. For Android. For Android, yes.

But I would push though that it needs to be something native to the device that's always omnipresent for it to really move the needle. So that's why I'm kind of bullish that it can only be the hardware.

Yeah, yeah, yeah, yeah.

Every time I hear about AI, they all say we're too early, right? I think I'm hearing the same from you, and every week feels like a year. Rapid growth. What are some biggest lessons you learned building Wabi and Cartesia in the last 12 months? Bonus points if you share the crazy moment.

I went first last time, so you—

Totally. Biggest lessons. Yeah, I guess I don't know if I learned anything. I think the thing that's interesting, I think, about for us at least is that we're building models and then building infrastructure to make those models easy to use. And so in some sense, because we're training models and we have an actual research effort. There are some comps against research companies that are now very large like OpenAI, Google, et cetera. And then there's a part of the work we do which is more infrastructure where it's quite different. And so, and that's really our product, right? So I think the challenging thing for us has been how to balance that, where to spend time because they pull you in slightly different directions. I think research is necessarily like a multi-year activity, you invest in it over time and then it pays out because you build something nobody else can build. At least that's how I think about it. And I don't think we're at that inflection point yet, at least for Cartesia, because we haven't yet built something that we think truly is game-changing yet on the research side. But that's really where we want to get to in the next 12 to 24 months. And then on the product side, then the balance is like, well, here's what customers want. Fuck research, let's go build the product. And I think it can be quite tempting to sort of draw resources away, but that's kind of the hard part about building this company. It's like you're sort of split-braining two things, which is also violating conventional advice around startups. And so we're constantly like, we're violating conventional advice around startups, maybe this is stupid. And I think that's the challenging piece for us. I think we would certainly go, you know, we're a 30-person team. If we abandon all research and all models, we would go way faster and just build product. But I think that's not why we started the company. And then in terms of other lessons, I would say I think one thing I realized is like, well, this is more of a team thing, but even with great people, it's really hard to just hand something off to someone and have them do it. It's just like you kind of just have to care a lot as the founders, just a lot to a degree that is surprising. Yeah. You really have to drive.

Yeah. Any crazy moment?

Crazy moments? Honestly, it was a pretty low-key, pretty— Yeah. Yeah. Pretty chill. Pretty chill. I lost 10 pounds. We were talking about this. Yeah. I didn't actually need it, but—

Well, you said, I think you said something meaningful, like—

I hope I did. No, I think you did though.

I think you said something about like something— I'm curious to ask you because you said it wasn't like a mind-blowing change. Like what is like the next milestone that's like a mind-blowing change?

On the research side? Yeah. Yeah, yeah, yeah. I think like, so we've done a lot of new work on our models. Our models are trained in a way that nobody's ever trained them before. For example, for speech synthesis, we're doing it in a way that's not been ever done, which is quite difficult. And the fact that it even runs in production is a miracle already. But I think the point is that we don't think speech synthesis is the sort of be-all, end-all, endgame application. One of the things that I tell people when they're joining the company is that we're basically a robotics company, even though we do not appear to be. Because sort of the things that we actually really believe in and want to build are highly efficient models that run on device that are multimodal. And I think the killer application for that is actually robotics. But I think that there's two ways to approach robotics. One is to say, I'm going to build a generalist robot. The other is to say, like, there's many challenges in on-device intelligence to solve first, and then you should go figure out how to build the application around it. And so we've chosen a path that I think is kind of strange, but kind of makes sense to me at least, which is mostly what I care about there. And so I think that's where I mean inflection points. It's like, how do we solve those bigger problems? Because that's what's satisfying on the research side. It's like, oh, we figured out how to do video understanding, audio understanding, conversation, all of this stuff, and it runs like a human brain on very little power. So human brains run on 20 watts of power, very efficient. Models are very expensive to run. So I think that's the contrast that we want to close the gap on. And we think there's 3 or 4 research breakthroughs that we mapped out that are required to get there. But I think the issue is that you can't guarantee research breakthroughs. You know what I mean? It's not like, hey, in Q1 we're going to do the first one. We're working on the first one, which we think will be very impactful in research, and then we'll sort of go from there.

And what was the question? I lost track.

So what's like some of the biggest lessons you learned building Wappi?

Oh, okay, yeah. Man, the hard parts.

You guys have time?

Sorry?

You guys have time? He's going to go on and on.

Oh, it's going to be a while. Yeah. The main things are, I guess, all the challenges I didn't expect. So prior to this, we were a startup doing AI note-taking for 3 years. We weren't growing a lot. And then all of a sudden, we were growing a lot. And everything was breaking all of the time. And I'll just purely start on infrastructure. We just had customers who just all of a sudden just wanted to rip 10,000 calls simultaneously. Well, how do you do that? Like, the infrastructure is so exotic that we run, and Nikhil's, you know, the mastermind behind all of it. But like, the short version that he tells me to say is we're running long-lived stateful jobs because everything has to run like one call per worker, and you can't throw everything up to serverless because they can't be alive for that long or something like that. See, he gave me the nod. And so, because of that, we had to, you know, rewrite our infrastructure from scratch like multiple times just in later of of last year because we had customers who would deploy, they'd have these big demos with all of their customers and all of a sudden the whole thing would just like shit the bed when they flew out 30 customers to like try their sales training product or like whatever it was. And so those were the things that I think hurt the most. And then eventually that problem becomes, okay, now how do you hire a bunch of people to then go build out this stuff that are also really good and down to join like a sketchy early startup where we're a bunch of like 25-year-olds. And like don't know what we're doing. And so then the challenge became, okay, recruiting a team. And it's like that— it is mind-blowingly hard to find like the incredible people that we were able to find on the team that were able to do stuff like put this event together in like literally 2 hours. And so once you have the people, then it's like, okay, now we have like 20 people. How the heck do we get all these people to do what we want to do like together and like coordinate them and make sure like everyone is aligned and knows what is going on. And we all now use tickets and boards and stuff like that. And now we're literally in the thick of that, learning how to manage people to do the thing that I want to do. And so it's been, I guess, a learning experience for me as a founder to try to figure out how to align a team of— even though they're incredibly talented and motivated individually, it's still on us to figure out how to get them aligned. And so yeah, those have been the 3 things.

I love that. And for folks listening, if you want to lose weight, build a startup. All right, moving on. You both raised significant amount of money funding last year. What are a few things you're planning to do in 2025?

With the money?

Cool office parts.

Swag.

Yeah, I think for us, and the reason that we raised our Series A is so we could start to get really awesome talent. I think part of it is about the money itself to invest in R&D and stuff like that. But it's also a signal to people that's like, hey, we're legit. We're doing something really cool here. You should come join our company because we're going to deploy this capital to do even bigger things at a larger scale than before. And so that's— that signal both to customers, both to potential hires, it starts to accelerate things. And so for us, that was pretty great. And now that we have some people, we now need to get more people and then grow the team probably like 3x by the end of the year. And so those resources are expensive as well. So that's really it. And then what are all those people doing? The big problem that we're trying to tackle that I should have also mentioned on the last question is model determinism. So now that we have these models, we need to make sure they actually do what they're supposed to do in a step-by-step fashion. And so what's missing in the models today? They kind of just like freestyle on like whatever prompt you throw at them. And so we need to build a system that allows for conversations with guardrails to be defined and actually work the way they're supposed to, not just like 90% of the time of production, but like 999 times in production for like the real enterprise use cases. And so, that's the real like hard kind of engineering problems we're trying to solve right now.

[Speaker:KARAN GOEL] I guess for us like we— so we did our seed in February, March, and then did a Series A end of November. And then, It just so happened that we announced our seed in November, and so I got congratulated on the seed, but I had just done the Series A, and I was like, well, actually, this is old news. But I think we are really, well, A, the costs break out a little differently because we do have to use GPUs, a lot of them. And so that's definitely one. One aspect that I think requires more capital to fund on the research side. And I think same for us, we wanted to just ramp on team. I think for us, actually we started out last year. I got married on December 30th. I got food poisoning first week of January, and then I came back to the US from India. And it was like we were 4 people. Basically working, and first hired my brother. He was our first hire, and then hired a few folks in, and we got going. And summer we were like 12 people, we had 11 interns, which was an interesting decision as well, which was mostly my doing. And half of those interns now work for us full-time. And I think post-summer we realized once we launched the product, like, hey, 10 people, is not— once you get a customer, it's like, well, oh, there's so many things you have to do. We didn't really know we had to do all of this other stuff like compliance and having infrastructure that actually works. So I think it was kind of natural that we just needed to grow and scale quickly there. And I think I also wanted to invest more in the research. And so that was kind of how we ended up doing a Series A. I think in 2025, it's going to be like a lot of— you're going to see a lot of releases from us on A lot more than last year actually on both the research models and product because I think last year was like very small team just trying to get things out. Our first product was basically built by the founders plus a few people, which is how it is in most startups. And so now I feel like we have more leverage. It really feels like, hey, to your point, there's so many great people that we have. How do we channel their energy and time into building really cool, interesting things that people want? So the resources certainly help. I'm also like a slightly Indian founder in the sense that I don't like spending too much money. So even on GPUs, I've had my team argue with me for 3 months they need more GPUs. But I've like— we basically have a util, and I monitor that util to check GPU utilization. I was like, no, there's no way you're getting more GPUs without utilization. So yeah, we try to be a little scrappy. I think it's important to like— Be conscientious when you're spending money. I love that.

As an Indian, I approve. All right, question for Jordan. As CEO of VAPI, a platform for developers to build voice agents, what advice do you have for someone who's building voice agents?

It's a good question. There is— I think a lot of people right now are scared off of building on the last mile to production, like within specific verticals. But okay, I was going to swear. Holy shit. I'll say it. Holy shit. It's really hard. It's like really, really hard to actually get a deployment to production with all of the nuances that are specific to like a specific customer. So for example, like we have this like medical practice group based in the Bay Area that we're serving, and they just want to do like appointment scheduling over the phone where like an AI agent or a person calls into an AI agent, that asks them some questions about their zip code and whatever, and then books the appointment. But then it's like when someone calls in and they say 4 PM, it's like, wait a second, what time zone are they calling from? And then now we have to fetch which doctors are available within this specific region and then read out those doctors' names in a way that is okay, because with accents and stuff like that and names, some of the text-to-speech models can say weird things that are not the most legible, and then picking up something as simple as having a transcription model understand your address. Like if you say like 123 Blumenfeld Drive, Blumenfeld is not a real word. You can't prime a transcription model with like every street name in the US, right? And so those nuances are the things that just completely block deploying within each of these verticals. So there— my advice or my comment is like, yeah, there's still a lot to do in every single vertical. Invoice, that is specific to each of these use cases. So there's still a lot of value in that bridging between the infrastructure layer and then the actual end customer. Nice point.

For Karan, so you kind of briefly mentioned about in this conversation that you are going to probably beat Apple at distribution.

I don't know if I recall saying that, but we can roll the tape again.

Talk to us about How? It's such a big statement, so.

I guess I'm not suggesting that we will out-Apple Apple, but I think that one thing I learned doing research.

Let me rephrase, like how do you take it mainstream? Mainstream, like.

Yeah, I think, well let me start from saying that I think one thing I learned doing research in a lab because I was doing my PhD before we started the company. Is that you can do very interesting new things in very small teams. Like teams of 1, 2 can actually do work that basically entire organizations do not do, even with billions of dollars. And the evidence is we actually did something that was new, that was interesting for once. Because research is really hard to be convinced that you did anything interesting. I've been doing research for 10 years. Probably one thing that I think was actually genuinely interesting that we did, which was all the work that we've done on state-based models, architectures, and so on. And I think that's sort of, to me, an indicator that there's a lot of power in small teams. So I don't think it's particularly intimidating to me that, hey, Apple is this huge company. They're building models. They've built phones. They've got the OS. They've got the hardware. You have to kind of start in one place and build something that people want at the end of the day. I think on the model side, especially for audio, it's really like how great can you make the experience of talking to a personal assistant or an audio brain basically. I think that is not something where you have asymmetric advantage if you're Apple. I think there's only a finite amount of capital required to do that, and maybe we will need to raise half half a billion more dollars to do it. But my point is it doesn't require a trillion dollars. I think what it does require is the right people to have the right ideas and to actually execute them really, really well with high craft. And so that's why I think I'm very optimistic actually that— and you see this even with models like DeepSeq and stuff, right? There's new folks on the block that you've never heard of that are doing crazy innovative work. That seems like it's outcompeting companies that have invested billions of dollars. So I think there's actually a very level playing field now in models, and that's particularly exciting because I think access to compute capital, et cetera, is obviously there, but also the fact that you can actually build amazing models without needing to be the first one to do it. There is some advantage in being second, third mover, right? Which is that you do get the sort of existence proof. But obviously we want to do something fundamentally new there. That's kind of my take there. I don't think we're actually trying to compete with Apple. I think at the end of the day, we are trying to just focus on— this is very much a research endeavor for us right now, which is like how do you build great audio-native models? And I think if we do a great job, then obviously the inflection point will be, hey, we have something that other people don't. Let's commercialize that as quickly as possible. A natural place could be to do like a personal assistant on the phone, among other things.

Build something people want. All right, let's get into the fun part.

Rapid fire for both.

All right, I'm going to start with you, Jordan. Coffee or tea?

Coffee.

That's the easy one, right?

I tried tea, it just— no.

No? I actually have the reverse answer. Sorry, not to— Yeah, yeah, yeah. You're a tea guy? Yeah.

How do you survive?

All Indians are tea.

No, right? That's right.

US or Canada?

You asking me or him? Oh, these are all for me. OK. Canada, yeah, for sure. Sorry. I'm Canadian.

I'm Canadian.

Jordan or Nikhil?

It's like English or Spanish.

Nikhil.

Best AI startup except Wabi or Cartesia that you love?

Cursor. Cursor, 100%.

OpenAI or Claude?

OpenAI for O3.

For you, best GPT model?

Best GPT model. 4.0. I'm still on 4.0. It's great.

It's low latency, man. It's good.

It's fast.

Mountains or beach?

I don't like either of them, but I'll go with mountains.

Best Twitter voice about voice AI you love?

Voice AI. God, that's a tough one. I'm going to have to come back to you on that one. It's not very rapid fire, I know. I feel like Alex from Hello Patient. He's very funny. Also very funny IRL. He's a bit of a troll. My Twitter is basically like all the trolls.

At this point everybody—

Maybe that's everybody's Twitter actually.

Biggest breakthrough in voice AI for you in 2024?

Definitely the OpenAI voice mode. That was great.

Yeah, the real-time API.

Yeah, it's pretty cool. Good.

We're going to take questions from the audience. Anybody has any questions for both of these gentlemen?

Hello. One thing I've been thinking a lot about recently, not just in voice but in basically every modality, is whether the future is going to look like big omni-models that do everything, or small models assembled into a sort of compound chain inference. Because if you look at VoiceMode, it does everything, but then the API is like, what, $18 an hour? When Whisper is at this point basically And speech synthesis is a couple bucks an hour. You only need a handful of L1 tokens in between. But you have to assemble everything, and maybe there's some trade-offs with those handoffs in that. So where do you see it going, to both of you? Omni-models or small models, small specialist models? I see. I think we're probably going to be on the same page. We'll see. But I mean, when you say small models, you mean like the cascaded system? Yeah.

Yeah.

So with the cascaded system, transcription, LM, TTS, the problem is each part, it's basically playing a game of telephone where like, let's say like a caller calls in and they're like really pissed off, but you can't tell by the words they're saying, but you can only tell by their tone. Like, oh, this is like the best experience ever or something. I don't know. All of that context is lost by the transcription model, and then the LLM will then just, like, multiply the loss, if that makes sense. And now it's like saying, "Oh, I'm so glad you had a great experience," and then it goes to the TTS model. The TTS model, like, warps it even further. And so this game of telephone is always going to be lossy in terms of the content. And then if you think about the actual turn-taking ability, the ability to backchannel and say things like, "Mm-hmm, got it," right? Like, all of those things cannot be done by a cascaded system. It's just input, input statement, output statement. But if you have a unified model, that model can both hear audio natively and produce audio natively without any loss in the system, and can also do things like natively know when a user's done speaking and then start speaking. We don't have to guess with, like, these specialist models that don't have full context over the whole conversation. And so to hit full human performance, it will have to be an end-to-end model. It just will have to wait until until folks like Cartesia get it small enough that it isn't $18 an hour.

Yeah, I agree. I think all multimodal models will be omni-models, but there'll still be a need for specialist models because of pricing. The omni ones will be more expensive because they're smarter or they do something a little bit more subtle than the specialist ones do. And probably that means they're a little bigger. They're like, going to be harder to serve, things like that. And I think what you said is basically the summary for why voice mode or end-to-end voice is required. And I think similarly, actually, I mean, I can go a little bit into the multimodal model building side, which is like, there's many ways to build an end-to-end voice model. And actually, there's ways to build it that look close to the cascaded approach of orchestration, but then you add on some end-to-end training on top of it. That's actually probably the simplest way to get there. I think the question is like how compact and great can you make it? And that actually is a fairly difficult challenge that requires working both on the data because you have to have all of this data that teaches the model how to do backchanneling and all this stuff, and you have to have very high-quality data. So I do think there's a huge challenge there. And then the hope is to then create recipes that allow you to do this for any language model. So the path to an Omni model is to take a language model which already knows how to reason and then turn it into a model that understands other modalities. That's at least the conventional approach. And of course, you can try to pre-train these from scratch, but you need huge amounts of compute. And then you're effectively doing pre-training on all the modalities together, which is quite expensive. So there's— what I'm trying to get at is there's a lot of— things to figure out in the design space of building OmniModels that is quite challenging. And we'll see how the iterations go. So for us, we're trying to build a version and there's lots of challenges there across all of these issues because this model is expected to also do things like transcription, et cetera, in the loop, right? It's basically understanding the audio and it should be able to decode those things pretty effectively. So yeah, I think it'll be interesting. I'm bullish. And I think it should be possible to get the price down a lot compared to this. And this is where the architecture side comes in, which is like Transformers are really expensive for multimodal data. I don't think they're the right approach. That's our whole spiel. And I think that what we want is actually models that can be efficient at any context size. So you can run a conversation for as long as you want. Actually, you should be able to talk to your bot for 365 days of the year, all day, every day, and it shouldn't run out of context or have any issues around slowing inference down because you're basically accumulating more and more tokens in your context window, which is the main challenge. Yeah, I think those are some of the things on the more researchy side.

And I'll also add, Right now, at least, the whole needing a unified model versus cascaded system— cascaded systems are actually fine for enterprise, like, right now, at least. But as the user behavior shifts, once these speech-to-speech models actually get deployed on actual mobile devices, people are used to talking to things and having it natively sound like people, that bar for performance is going to go up, and so then the enterprise gap is going to grow. And so eventually enterprise will have to move up to speech-to-speech, but the pain is not there. Yeah, already generative cascaded systems are already a huge step up versus the IVR systems and all that. All right. I really like both of your answers because larger, more expensive, harder-to-solve models are great news for me. Thank you, based on.

Next question.

How is VAPI thinking about— I'm just trying to imagine what on-device means for VAPI.

Oh, man. I think that's a great question. I don't think we— I don't see a path on the roadmap for on-device for us right now. I think our focus is like where on-device won't be able to exist. So when you call into like American Express or something, like, well, that just can't be on-device. Like, we still think about the phone phone line as like the universal API between people and businesses. And that API is here to stay, at least for the foreseeable future as far as we're concerned. And that API unfortunately cannot run on device. And especially at the scale that these enterprises want to run at, like it has to run in the cloud.

Yeah. So I think about a third of the latency is coming from Twilio at this point.

That's a very astute observation.

For these scenarios, I mean, how are you looking at just being able to kind of keep up with the web native kind of quality experience, but when there is a SIP server and a bunch of Twilio crap in between?

Stay posted.

Oh yeah, we're working on it. That's why, as part of infrastructure, we also invest in telephony infrastructure to make all that shit faster, running it ourselves, because Twilio's a little slow sometimes. But then generally across the board, We do see a trend of the transcription models are going to be getting faster in the next quarter. The text-to-speech models are definitely getting faster per unit of performance, and the LLMs are getting faster too. And so you get to save on latency in each part of the pipeline, especially with speech-to-speech. It's just going to be so fast, like 300 end-to-end.

So yeah.

Question from Karan Goel.

Actually, great model though.

You're one of those people who are trying to Switch as well from ElevenLabs to VeriSham, not yet, but might get there. The question is, like, how do you—

because there's a lot of speech models which are getting smaller, and then there's a case to be made if you're doing voice agents that you can go on-premise.

So what's your take on, like, an on-premise versus still going with an API as well in terms of latency being the last mile?

Yeah, so I think two axes there, like, one is, like, where do you deploy? And the other is possibly model size, like smaller versus larger model. I think for us, like, we offer deployments everywhere, like literally everywhere. We have our API, we do containerized deployments in Cloud VPC, and then we also do on-device deployments. And so the goal is to just be able to run the same model with the same quality regardless of the environment in which you're deploying. And I think we're actually close for text-to-speech there. Within the year, we should be able to run it anywhere with the same performance. And then the other axis on latency or speed and things like that, I think we're probably going to have this split of having a very fast model offering that is really good for voice calling. And then a higher quality one that's much more well-suited for creatives because it's like more prompty and things like that. And what we're doing is trying to shrink the latency down for the one that is meant for voice calling. So as an example, I was shown a model last week internally that is doing 25 millisecond latency. It's very fast. And so we're now getting to the point where like you can basically almost zero out the latency for speech synthesis. We actually think we can get it to 12 milliseconds, but it will take a little bit of time. But we know how to get there basically. And I think for users, I don't think 25 versus 50 is going to make a huge difference because network latency is going to be still part of the equation. But I think the fact is those will get really fast for the same quality and within this year. And so that will just make it really easy because basically for us it's going to be like you're going to come up, come to our website, you're going to find our Turbo model. And it's gonna be super fast and you can just put it anywhere, um, that you want. That's kind of the goal.

Henry? Really, I promise this is not a lame question. Uh, he's a bad PT.

When is your earliest guess for when end-to-end, uh, uh, turn-taking for AI will be human level or better, so 300 milliseconds or less? End-to-end turn-taking. So we're mining from stop of Hugh's speech to start of Guy's speech, all that was? Yeah, I think it's already 300 or less, I would say. You can run it in, you know—

It is possible technically.

Technically possible. Yeah, technically possible to do it. So I would say last year. The average. The average?

Yeah.

And average, you mean like you're on the— other end of the application basically. Yeah. I think this year, yeah. It should be possible to have like under 200 reliably.

But then of course, when you try to crack like the 800 milliseconds sound barrier, then the turn-taking needs to be like incredibly accurate or like every time you pause it just like, uh.

Right, right. Yeah, like I guess the thing is when should you not be 100 milliseconds and when should you actually be pausing for 800 milliseconds?

100 is like really, really tight. I don't think I'd ever want something the response.

Yeah, you definitely don't need that. Yeah, yeah, yeah. Yeah, I think actually the problem is can you pause in the way that's most natural for where we are in the conversation, which I don't know the answer to that one.

All right, that's okay.

One more question for Jordan. How do you think about the trade-offs between, let's say, building something a platform that's more self-serve, especially as that space gets more crowded, and building an application for your customers, for your large customers, for your first customers? Yeah, that's like the big existential question. Yeah. So the way that we view it is we have— so our self-serve has 100,000 developers on it at this point. And so that community is vibrant. In the Discord, it's like the United Nations. There's different channels for different languages and all sorts of stuff. All these people trying to, there's an economy in there, people trying to make money off each other. It's great. So that community is actually our secret weapon where everyone in this community then goes to join a company and then brings BAPI in their little toolkit and says, hey. Then when senior management goes, hey, we need to deploy voice AI in our customer service thing, The little intern at the back is like, "Yeah, I used VAPI for like a hackathon project. We should try it." And then the senior actually takes it seriously when he sees the docs and it kind of like bubbles up from there. So it's kind of like the self-serve is our like kind of growth flywheel that then drives our enterprise business.

[Speaker] Nice. [Speaker] Yeah.

[Speaker] All right, guys. That's a wrap. Appreciate you, everybody. And thanks, Karan Goel.

[FOREIGN LANGUAGE] [APPLAUSE] Thank you, everyone.

That was great. Thanks for having us, y'all.