S15E1: Building the Infrastructure for Voice AI with Russ d'Sa, Co-Founder and CEO of LiveKit

It turns out that, like, if OpenAI is going to build a computer that is increasingly becoming like a human, you're probably going to interact with that computer in human-like ways.

Welcome back to a new season of The Room Podcast. Hi, Mads.

Hi, Clauds. As you all know, we are opening the door to the room where it happens for today's technology founders.

I mean, I can't believe this is Season 15.

Crazy.

Probably one of our most exciting seasons telling stories up and down the stack.

After 6 years in the room, we're just delighted to be back together in San Francisco more and more, both being based here live from Vercel HQ. I think you have an update as well.

Yes, I'm also now at OpenAI.

Wow.

Which is fun, pulling us deeper into this revolution we're seeing globally.

Thank you everyone who's been listening to us since the beginning, 6 long years ago. And for those who are new here, please like, follow, subscribe wherever you listen to podcasts Or of course at theroompodcast.com. Today on The Room Podcast, we sat down with Russ Dassault, the co-founder and CEO of LiveKit. LiveKit is the company that built the open-source infrastructure layer powering real-time voice and video for AI applications, becoming the go-to plumbing behind the next generation of apps where AI agents actually talk and listen.

And now a message from one of our sponsors, Rippling. One thing we constantly hear on The Room is that grid operators don't just need more data, They need a faster way to turn that data into decisions. As a founder, I know how painful it is to wait days or even weeks for a report while a real business problem is unfolding. That's what makes Rippling AI different. You can ask Rippling AI, is our sales team at risk of missing quota? And Rippling AI instantly pulls real-time data from Rippling and your CRM into presentation-ready dashboard data showing quota attainment, headcount trajectory, and monthly revenue to quota by But it doesn't stop at the insight. Rippling AI can act on what it sees, recommending and building an automated recruiting workflow when open roles and slow time to fill are putting quota at risk, all while following your existing permissions and approval processes. Question answered, action taken, problem solved. Ready to rule your business? Head to rippling.ai/theroom and sign up for exclusive access.

And now a word from our new sponsor, Gradial. Whether you're building a company, launching a product, or growing a brand, you know the idea is just the beginning. And marketing's bottleneck has never been creativity. It was always the work it takes to get an idea out the door. Gradiel handles the operations so that marketers can stay on craft. An agent will learn your process, work across the systems you already use, and delete point solutions you don't need. Designers will ship on-brand pages without counting prompts or tokens. Product marketers will launch across markets in days, not weeks. And digital experience teams test customer journeys, catch what's not working, and fix it All before launch. Gradio learns that your brand, business, and process and brings that continuous context to every workflow. Every task gets done your way with brand, quality, and approvals built in. No more token maxing, slop, or runaway AI bills. More time to think bigger and create the impact people remember. Gradio, the system of work for marketing. Ashurst Perkins Coie is a leading full-service international law firm known for providing high-value strategic solutions for startups and investors. Lawyers at Ashurst Perkins Coie support the most innovative entrepreneurs and investors in fast-moving and high-growth sectors across the country while addressing the myriad of legal needs that arise when fundraising and scaling their businesses. The firm's tailored legal services are informed by understanding of each of the client's discrete needs. To learn more about Ashurst Perkins Coie, visit their website, ashurstperkinscoie.com. First of all, Russ, thank you so much for joining us today in the room.

Thanks for having me.

We're delighted to have you.

It's great to be here.

We'd love to start with the beginning, and we do this with all of our guests, just to kind of hear more about where you grew up and then, of course, how that shaped your view of the world.

Yeah, so I grew up— I'm Canadian, so I'm nice. No.

No accent.

No accent. That's right. It got hammered out of me when I came to the States and everyone made fun of me for saying eh after every sentence. But no, I grew up in Canada. I grew up in Canada in a small town outside of Ottawa. And my dad was in tech. So he was an electrical engineer in England. Eventually him and my mom made their way to Canada where I was born in Toronto. And then after about a year, my dad moved— the family, we moved to Ottawa for a job he had. And so I grew up there. surrounded by computers. My dad has, like, a pretty crazy story. He went from, like, semiconductor industry, GPUs, and then DSL and high-speed networks, and then into flash storage. So he's kind of, like—

Oh my gosh.

Worked across all of these foundational kind of, like, hardware revolutions that have happened over the '80s and '90s. And then so I was growing up, you know, around him, and he's, like, one of my best friends and my hero. So, just surrounded by computers. My dad eventually got a job out in Silicon Valley in the early '90s, and he went out himself to Cupertino.

Wow.

And so he was in Cupertino flying back to Ottawa every month for about a week. And after a couple years of doing that, he decided, hey, you know, let's move the whole family out. And so we all came out here in the mid-'90s. And then, you know, I've been kind of growing up in Silicon Valley ever since, and my dad always took me to things. Yeah, I think I was the youngest person at CES, or it wasn't— was it CES? I think it was CES. Youngest person at CES back when I was like 14, 15 years old. And yeah, I think like I've always been fascinated with like programming and wanting to make video games. I think was a classic origin story for a lot of programmers in the '90s, growing up in the '90s.

Totally.

Yeah, I've been steeping in this stuff. I kind of feel like I was always gonna do this no matter what.

That resonates so much. My family moved from Atlanta, Georgia to Seattle when I was 4.

Okay, yeah.

And my dad also worked in tech. And just the fact that as a 10-year-old, I knew what the cloud was and kind of that concept of, like, you know, Seattle being the cloud capital of the world with AWS, lots of tech companies.

Was your dad at Amazon or Microsoft?

He was at Madrona, which was an early investor in Amazon.

Oh, I know Madrona.

Yes. And that to me was just like what every dinner table was, like talking about great companies.

Crazy. Madrona passed on our Series A, so that sucks for them.

Oh, yeah, that does definitely suck for them, because we're going to get to some big numbers in a second.

I know, yeah, I know, I know.

Well, I, you know, curious for you then, having that steep and, you know, knowing obviously so many people around the corner in the industry, did you yourself always think you were going to become a founder?

Definitely. I think, and not necessarily a founder in, like, the sense of how it's manifested now, but I think I always wanted to start my own company, but mainly around things like video games.

Okay.

So when I was— it's funny because my dad was in startups and all of that technology, but where the real kind of motivation came from was when we were flying to California to visit him for the first time after he had moved. I watched this documentary on the airplane.

Okay.

This was like, you know, in like '92.

I was gonna say, like, on what, the TV?

Yeah, they had a screen. They had a screen, but it was like the shared— I don't know.

Shared screen?

No, it definitely wasn't the shared screen because nobody else would have wanted to watch this thing. It was a documentary about John Carmack and John Romero.

Nice.

So John Carmack, I think more famously now for Keen Technologies, which is the AI company, and then Oculus, where he was the CTO, but He was also one of the founders with John Romero of id Software.

Oh, interesting.

And id Software, you know, they made Doom as one of their most famous games and Wolfenstein as well. But like, I played those games, you know, growing up. And so there was this documentary about the two of them, and they tell this story about how they were working at this company, and then in the evenings, you know, they wanted to work on their own stuff together. So in the evenings, they would unplug the computers, put them in the back of their pickup truck, drive the computers home, work all night on their own stuff, and then take the computers back in the morning before anyone showed up and then plug them back in. And that wouldn't fly, you know, IP rights and all these days.

Not today.

But I just thought it was kind of amazing how these 2 guys, nobody's ever heard of them, could like type words on a screen and influence the world You know, hopefully in a positive way, just from their garage or from their house. And so I was really enamored with that idea, that ability to have this outsized impact as an individual. And so it was kind of from that moment that I decided, okay, I'm gonna— I wanna be a founder one day. I wanna create something myself.

That identity was steeping in you.

Yes. Yeah, for a long time.

And I'm— well, have you read the book Tomorrow Tomorrow Tomorrow?

I haven't.

Okay. It's worth a read. It's kind of a normie gaming book, though, I will say.

Oh, okay.

But they're about to make it into a movie.

Oh, yeah.

And just kind of what you're describing of, like, kind of how you fell in love with the whole ecosystem I have to know what that first company was that you got into YC with. Was it a gaming company?

No, actually, it's more embarrassing than that.

Oh, please.

So back then, there was this moment where webcams were being built into these white plastic MacBooks.

Okay, yeah.

So all these students— when I went to College of Davis, a little bit up north from here, and there were people that were buying these white plastic, you know, kind of laptops, and they had a camera that got built in. And it was the first time that, like, a webcam had been integrated into a portable computer that you could take with you anywhere. And so I thought to myself, I'm like, oh, wow, you could, like, broadcast yourself with this, you know, this camera here anywhere you are. Initially, the idea was, well, why not livestream yourself everywhere? And it was right maybe, like, a couple months after that Justin.tv came out.

Oh, interesting.

Which is so funny because Justin is one of the first checks into LiveKit.

Nice.

So it's just like this crazy weird full circle. But Justin.tv had just come out, and I'm like, okay, well, can't work on that idea. You know, and it's a YC company too, so can't work on that. Maybe instead we'll work on, like, something where you can meet anybody on the internet live, right, and chat with them. So yes, it sounds like Chatroulette. It is Chatroulette in a lot of ways. In a lot of ways, Chatroulette many years before Chatroulette. But the angle of it was around speed dating.

Okay.

So we needed something that people could kind of, like, see as, like, a purpose. Or at least we felt like chat didn't really have a purpose, but we felt there needed to be a purpose for people to even do this thing. So we're like, okay, well, how about dating? Because that— there's this analogy in the physical world of people doing that. Maybe people will do it over the internet instead. Yeah.

What year was this?

This was 2007.

I feel like that was a novel time to be thinking about how people might do this.

Oh, very early. Yeah. Literally nobody was doing— except for Justin.tv, nobody else was doing video at scale in a browser.

Right.

I mean, it was almost impossible to even do it.

Very difficult. From an infrastructure perspective, extremely difficult. I mean, when I think about what we do now, and I think about what the tools were like back then, yeah, it was dire straits out there.

Well, it didn't take off to the levels that you would have hoped with that first company.

It didn't.

And ultimately you joined Twitter, correct? Tell us a little bit about—

No, no, actually, I joined 23andMe first.

What? Ooh.

Yeah, yeah, yeah.

OK. Well, tell us about that transition, then.

Well, before I did YC, I had gotten a job offer. So this random company of 5 people called 23andMe had sent me an email, a recruiting email. And so I had happened to read about 23andMe on— there was this blog, like a tabloid blog called Valleywag.

Oh, yeah.

That's old. Old school. Valleywag was writing about how the founder of Google's wife, or they weren't married at that time, but girlfriend is, like, starting this company, this genetics company. And so I had heard about it, and then I got this email. So I'm like, oh, okay, it's interesting. I at least read about it. So I went and did an interview loop with them, and I got offered a job. And then I also got into YC, like, maybe the next week.

Wow.

And so, you know, this was my moment to kind of finally make my dreams come true, start a company.

Of course.

Follow in my dad's footsteps. So I ended up going and choosing YC instead. And then after, you know, about a year or so, my YC company didn't work out. I went crawling back to 23andMe, and I think they were about a 30-person company. And I emailed them and I said, hey, are you still looking for folks? And they were like, yeah, yeah, yeah, come do an interview. And so I went and I interviewed again and got the job. And that was my first job after my first company. Oh, wow.

Oh my gosh. Well, I mean, really different overlapping ecosystems, right? Like, obviously everything at the forefront of what's happening on the internet, but yes, completely different verticals. So yes, how did that all come about? And then ultimately, yeah, like finding yourself in the Twitter ecosystem.

I think that I, um, I was drawn to 23andMe because they, you know, have on my LinkedIn, I talk about how I like, I always work on these ideas before, uh, too ahead of their time. And I was attracted to 23andMe because I had actually personally I had never taken a biology or genetics course ever. They somehow fooled me into taking this integrated science thing in high school and not taking biology, which I hear they're still doing in schools. So I had never taken biology, and I have long thought that eventually technology will go from kind of its own category into something that permeates across every discipline. And where every discipline can get leverage from technology in some way. And so I think this cross-disciplinary combination of, like, genetics and technology software was really interesting and compelling. And I didn't know anything about half of it, right? And I probably didn't even know even about half of technology either at that time coming out of school. But so I went there, I was attracted for that reason, and I went there and I thought I would be able to learn a lot. I was really early in my career, so optimized for learning, and it was an amazing experience to be there. And so, yeah, I'm really grateful for it as my jumping-off point for my career. What ended up happening with 23andMe is that 23andMe, they went through, they've been around for a long time.

Sure.

I joined the company in 2008. They'd been around probably since 2006. And then they eventually, I think, filed for bankruptcy. During my time there, between 2008 to 2010, they had gone through some challenges because the consumer acceptance of genetic testing, it was very early pioneers, right?

Right.

And so the acceptance wasn't there. There were a lot of privacy fears. Eventually that FDA kind of mandate came down that they had to work with the FDA on for many years. But during that time, 23andMe kind of struggled to figure out whether they could build a really solid business in the consumer space. And there was always this temptation, as their genetic database grew, there was this temptation to partner with Big Pharma on, like, drug research and things like this. And Big Pharma kind of has a reputation.

Yeah.

And so 23andMe, during these struggles, they had a couple of rounds of layoffs. And I was a little bit nervous. A bunch of my friends had gotten laid off, so I was pretty sad about it. And then they started to talk about this shift to maybe working with big pharma. And I was like, oh, I don't know. I don't know if that's really what I want to do. I kind of want to, like, continue working on consumer and internet. And one of the people that had gotten laid off from 23andMe was this recruiter who became the head of recruiting at Twitter. Wow.

Fascinating.

And so he had been pinging me, sending me these emails to my personal email address for like 6, 7 months, trying to get me over to Twitter. And I was just ignoring him. I'm like, ah, forget it, forget it. I'm just not gonna respond. And then once this big pharma thing happened, I'm like, okay, well, let me go see what's going on there. And so I went and I did some interviews, and I did well there. And so they offered me a role.

In software, on the software side, on the content side?

Yeah. So I initially tried to interview for a backend systems role. And they said, well, you're going to interview to be the second frontend engineer at Twitter because we don't have anybody that knows JavaScript here.

Oh my God.

It's kind of funny that I'm talking to Vercel people. They only had one person on the entire team that knew JavaScript. And so I knew JavaScript, but not really as an application developer knows JavaScript, more as like, quick scripts in the browser to maybe make something interactive. Nothing in-depth.

A gamer mindset.

Yeah. I ended up going there thinking I would learn a lot from this person who was the one frontend there. He taught me everything I knew about JavaScript and I became an expert. That was my time there or how I ended up there.

That's a storied early career, honestly. Starting at so many just category-defining businesses across so many different frontiers.

It's incredible. I mean, like, another kind of crazy story connected to that is I had a friend who I'm actually now trying to get over to LiveKit.

Nice.

Top circular. Yeah, I'm competing with Anthropic for him.

I'm not gonna say his name.

So I had a friend who, at Twitter, we actually built the first single-page application basically ever.

Okay.

In JavaScript. And it came from work that I did there. So, like, I— when I got to Twitter, we shipped this thing, this project called Hover Cards, in 3 weeks. And Twitter was notoriously slow for not being able to ship anything. Like, the product was just in stasis. And so me and this client, the other JavaScript person, shipped this thing. And the team was, like, so impressed by the speed that they're like, okay, we're gonna move you to the platform team now, and you're gonna work on this thing called Twitter@Anywhere. And Twitter Ad Anywhere was like Facebook Connect, but for Twitter. So you could bring in tweets and, like, linkify everybody's, like, @handles on their own websites. So you put this on CNN, and, like, the whole thing, like, changes to be all kind of, like, Twitter-infused. As part of doing that, the thing you had to learn at that time doesn't really work anymore with all the browser security that's come out since.

Darn that security.

Yeah, right. But there's this thing called an iframe, and it allows you to effectively embed another page within the webpage. And you don't have to necessarily display that page. You can actually, like, just bring in JavaScript and then manipulate that page through an interface that exists between the embedded page and the outside page. So I learned this, like, dark art of iframes because it truly was, like, this very esoteric thing. And so I mastered it, and One of the things I did for that project was I had to work with the Twitter API from JavaScript.

Wow.

And so I built this SDK that we could use for that project that used something in JavaScript called the promises model. Nobody was doing this at the time. I won't even describe what the promises model is. I'm sure some of the folks listening to this might know, but built this SDK, very rich and expressive. You could easily create Twitter clients using it and pull in information from Twitter. And so after that project finished, they were gonna grab me and take me back off of platform and put me on the web team again to rebuild all of Twitter and redesign it. Like, the modern design of Twitter today actually came from that project.

Wow.

They were gonna pull me off and Dustin, my coworker, off and work on it, and I didn't wanna leave. So what I did was I did this cheeky thing, which was, There were designs on our, like, on our portal, on the design team's portal at that time. I was going to say a wiki. I'm like, people don't use wikis anymore. We had this design on the wiki of what that website was going to look like, the redesign.

Yeah.

So I went and I took my API and I built the whole thing in, like, a couple of days to illustrate how fast you can do it with this new thing I built. And then I went and presented it in front of the whole company and said, hey, look, you don't have to take us off the platform team now. We built all of your— we did all your work for you. I was in my 20s. I was a jerk. But we did it all for you, so leave us on the platform team. And then they ended up saying, OK, well, you know, instead of a redesign of Twitter, we're gonna rebuild all of Twitter. And so they locked us in a war room.

Oh, wow.

We rebuilt all of Twitter. We launched it as new Twitter. It was this awesome thing. And there was this guy on my team who was just brilliant, but he was more like an architect in that project. He came in a bit later, and so things were kind of moving. He had looked at my API and he had looked at what we were doing in the Twitter redesign. He's like, hey, I have this idea. This idea is I think we can create this framework that will bind the UI to data on the backend. So when you update data on the backend, the UI just magically gets updated. He's like, I'm going to call it React.

Shut up. I knew you were going to say that.

So we're working on this thing and I'm like, I don't quite understand what he's saying because he's just way smarter than me. But I was helping him work through it. Then I left Twitter and he stayed there for a few more months, was still working on React, but it was a side project for him. And then Twitter decided, I don't want to do anything with this. And then he started, he got connected to the people at Facebook.

Naturally.

And they were collaborating and then he's like, yeah, you know, they had their own project that was very similar to this. And he's like, you know, you guys, why don't you guys launch it? And they're like, okay, great, but like, can we use your name? And so they took the name React and that's how it became React. It's like lore. Yeah, I know, it's wild.

It's like sacred we're doing this inside of the Vercel office today.

Well, that's why I brought up the story. Yeah, it's epic. Where else am I going to tell this story?

Well, I think it's a perfect segue to start talking about LiveKit. You've been at the frontier of so many sort of new primitive innovations from back in the early-ish 2000s, 2010s. And now we're kind of on the precipice, or we're in the midst of, like, another huge stepwise function in technology. We are.

Yes.

So let's talk a little bit about LifeKit. Take us back to the original aha moment.

So there were 2 aha moments. I think there was an aha moment for starting the company, and then there was an aha moment for what the company is today, or at least the product that we build today. And so the first aha moment was the pandemic, right? You're at home.

Yep.

Can't leave your house. And so you got to, like, be on Zoom with everybody. The aha moment there was that the technology that underpins Zoom is actually also very esoteric technology.

Yes.

There are maybe, I kid you not, like less than 500 people in the world that have expertise deep expertise with the technology that kind of powers or makes Zoom possible, being able to move photons of light and vibrations of air anywhere in the world with less than, you know, a few hundred milliseconds of latency. And so Zoom, the technology around it, there's both technology software that governs it, and then there's like protocols. And the protocol for how to do it was open and out there, just not leveraged by many people. What Zoom and Google Meet and Discord have done is they've built— basically spent 10 years building technology around that protocol to get it to scale, because the protocol itself wasn't developed for scale.

Sure.

And so they kind of, like, have these techniques to get it to scale. And I was working on— I had actually, like, sold a previous company to Medium, the blogging platform. My co-founder and I, we were now there, you know, went from a 13-person to a 300-person company. And pandemic hits, stuck at home using Zoom like y'all every day. And then Clubhouse comes out into alpha.

Nice.

And I'm like, man, this thing is so awesome. I'm like staying up late, like just lying there listening to Clubhouse, you know, people talking.

We had Clubhouse. Yeah, for sure.

And, you know, there was something about like just how it made you feel a bit closer to people in a time where, like, you just— you didn't have that human interaction that we had had before or enjoyed before. And so I thought it was really cool, but I wanted it for my company. I wanted it for Medium. I wanted to feel like I was still passing by colleagues in, like, the cafeteria making coffee.

Water cooler.

Yeah, I called it that. Yeah.

Really?

Yeah, it was called Water Cooler. No way. Shut up. So as a side project, I'm like, I'm gonna make a Clubhouse for companies. But the part I don't know how to do is I know how to build apps, but I don't know how they're doing the audio video streaming, at least in 2020, 2020. I know how it was being done back in 2007 with my YC company. And so I learned about all this stuff, and I pinged my old co-founder. I said, hey, let's— can you help me with the infrastructure so I can build this thing? And, and so we started building it. The water cooler thing, it didn't end up being that great of an idea because it wasn't really about, like, Clubhouse for companies. It was more about, like, how do you actually build company culture in a hybrid remote workplace, or at that time remote? And that's, like, a different problem. Like, yeah, Clubhouse for companies is just a technique. Audio rooms is just a technique, but it's not the only technique. And so I wasn't super passionate about that problem, but a lot of friends had heard about LiveKit, or had heard about this thing we were building. It wasn't called LiveKit then. And they said, hey, can we get access? So we created a repo and a Slack instance and just invited our friends to it, and it just exploded. So it was like a top 10 repo on all of GitHub for every lang— across every language for like 6 months. And we had all these people coming to the Slack instance, like, asking us for things.

Right.

And so that's when we left Medium and we said, you know, we're gonna kind of focus on LiveKit and build this out. And so that was the first aha moment that people really wanted this thing. Right. Then there's the next aha moment, which is we launched it, first release of open source, and like a public, large public company, $100 billion-plus public company, tried to buy the open source project the first day.

Okay.

Damn. Good signal. And we're like, okay, this is crazy. And then like, yeah, great signal. And then the next week, like Reddit and Spotify and Oracle and X all ping me and were like, hey, we're deploying this thing, you know, how do we make this thing work? How do we scale it? And we're like, okay, what's going on? So we met with them. They're like, hey, could you give us like a cloud version of this? Like, make it so it scales to the moon and it has 5 nines reliability like Zoom does, but at the API layer. And yeah, we want a global edge network with the lowest latency possible like Cloudflare has. And so build that. And if you build that, then like—

We'll pay you a lot of money.

We'll buy you guys. I mean, we'll buy it from you guys, like the product. And so that's when we raised a round and we started to work on it. We spent 2 years. End of 2022, we put it out, super proud of it. 3 weeks later, ChatGPT comes out.

Shut up.

And I'm like, man, I'm— this is like texting with a person.

Yeah.

Forget texting with a person. What about if I could talk to the computer, talk to the AI? So we took LiveKit, built a demo, put it out there ready to go viral. on X and, and like only 100 people liked it. So I was so sad and I forgot all about it until 5 months later. And then OpenAI emailed me and said, hey, we signed up for LiveKit Cloud in secret with a personal Gmail address 3 weeks ago, and we've built a voice interface to ChatGPT on top of LiveKit Cloud. On your LiveKit domain. And yeah, we're really excited and we want to talk commercial. And so that was the next aha moment. It was, all right, like, it turns out that, like, if OpenAI is gonna build a computer that is increasingly becoming like a human, you're probably gonna interact with that computer in human-like ways. And so all this technology we've been building to connect people together during the pandemic can actually be used to connect a person to a machine In this new world that we're moving towards. So that's kind of the second aha moment.

That's incredible. It's so cool to hear just sort of how strong the product-market fit pull was in a way that's a little bit unique to businesses, right? Like a lot of startups, it's like we solve this problem, we fundraise, then we try to go get customers.

Yes.

It was kind of the inverse for you guys.

Yeah, it was very accidental. I actually gave a talk last year at the, at the Web Summit. I was talking to Ben Thompson in Vancouver.

Canadian.

Yes, there you go. Have to. So Ben Thompson from Stratechery, he was— he calls OpenAI the accidental consumer company because they never meant to create ChatGPT or they never anticipated it would turn into this phenomenon. And then I joke that AliveKit is the accidental AI company because when ChatGPT came out, everybody was like raising money for AI.

Sure.

And I was like, yeah, I'm kind of sad. Like, I really want to be part of this AI thing, but I've raised a bunch of money. I can't really— I don't even know how we would pivot this thing to AI. So I guess I'm just gonna have to sit this one out.

Yep.

So I was feeling the FOMO, but the thing I refused to do was to, like, reframe our company as an AI company. And then, like, the OpenAI thing happened.

Lo and behold.

I'm like, oh my gosh. We have an AI thing.

When you build a sick product.

Yeah. So the accidental consumer company turned Livekin into the accidental AI company.

And it's safe to say you're not too early on this one. I think you're actually right on time.

Yeah, this one I would— yeah, this one. Well, I don't know, maybe 5 months too early because of that demo. But, but yeah, the stars aligned this time and I'm grateful for it.

Well, speaking of investors, you've raised from incredible groups. Yeah, it's been great. Most recently, a $100 million Series C from Index alongside Redpoint, Ultimeter, and incredible other investors. Who was the first person to say yes to investing in LiveKit? One thing we constantly hear on the room is that great operators don't just need more data, they need a faster way to turn that data into decisions. As a founder, I know how painful it is to wait days or even weeks for a report while a real business problem is unfolding. That's what makes Rippling AI different. You can ask Rippling AI, is our sales team at risk of missing quota? And Rippling AI instantly pulls real-time data from Rippling and your CRM into presentation-ready dashboard data showing quota attainment, headcount trajectory, and monthly revenue to quota by region. But it doesn't stop at the insight. Rippling AI can act on what it sees, recommending and building an automated recruiting workflow when open roles and slow time to fill are putting quota at risk, all while following your existing permissions and approval processes. Question answered, action taken, problem solved. Ready to rule your business? Head to rippling.ai/theroom and sign up for exclusive access.

And now a word from our new sponsor, Gradiel. Whether you're building a company, launching a product, or growing a brand, you know the idea is just the beginning. And marketing's bottleneck has never been creativity. It was always the work it takes to get an idea out the door. Gradiel handles the operations so that marketers can stay on craft. An agent will learn your process, work across the systems you already use, and delete point solutions you don't need. Designers will ship on-brand pages without counting prompts or tokens. Product marketers will launch across markets in days, not weeks. And digital experience teams test customer journeys, catch what's not working, and fix it all before launch. Gradio learns that your brand, business, and process and brings that continuous context to every workflow. Every task gets done your way with brand, quality, and approvals built in. No more token maxing, slop, or runaway AI bills. More time to think bigger and create the impact people remember. Graydeal, the system of work for marketing. Ashurst Perkins Kewey supports the most innovative entrepreneurs and investors in fast-moving and high-growth sectors, addressing their myriad of legal needs. But the firm doesn't just provide end-to-end legal and business counseling to its startup clients. It also facilitates introductions to key advisors and sources of capital. Ashurst Perkins Coie's interactive website, Startup Percolator, offers access to programs, resources, and rich dynamic content designed to assist entrepreneurs in their startup journey. To learn more, go to startuppercolator.com.

The first person that actually didn't say yes to investing in LiveKit when we didn't actually ask him, he actually So you said yes to the investor? We said yes to him. That one's also kind of a crazy story. He runs like a— he runs kind of like an incubator/fund. He's also a founder. His name's Furkan, and Furkan is the founder of Founders Inc. They have like a whole giant warehouse in Fort Mason, and he He was in like a WebRTC Discord or Slack or something, and my co-founder was in there too when we were first learning about this technology, trying to power my water cooler app. And my co-founder pings me and he's like, hey, by the way, this guy Furkan, if we ever start a company around this, he said he wants to be the first check-in. He had sold a company to AWS and was leading Twitch on the streaming infrastructure side. And so he knows about this problem firsthand. He's really experienced with it. He's done a lot of video and audio streaming. So he wants to invest because he knows how difficult it is to do. And I'm like, Furkan? I know a Furkan. I'm like, who's this guy? And so he sends me a link to his profile on LinkedIn. And I'm like, dude, I used to get drunk with this guy.

Oh my God. How many of them?

All the time in college. This was one of my brother's friends. At San Jose State, like, this guy became an engineer? Turns out he's actually a super smart engineer, but I just wouldn't have figured him for one. Pegged it. You know, people evolve.

They surprise you.

They definitely can surprise you. And so just kind of a funny— that was another full circle thing right there. But Furcon was the first one.

I mean, I love this thread of how relationships keep coming back around. surprising you in new ways.

When you're in the Valley for long enough, you definitely accumulate quite a lot of stories, and you run into the same set of folks.

Totally.

Yeah.

Cool. Well, founding journeys, of course, are never linear. I mean, your path to what LiveKit today was not linear. But let's say we zoom into the last year of scaling LiveKit, taking it to this new level. What's a moment where something didn't quite go as planned?

I don't want to say that it didn't go as planned because I didn't plan it. I think, like, I didn't intentionally try to do everything I could to evolve it. But the thing that I wish we had done differently maybe is a way to answer that for the last year. Last couple of years, there's definitely things that didn't go as planned, mainly in the form of outages. Right. You hit these tranches of scale and, you know, there are things that just break.

Yeah.

And in some ways, it's a rite of passage, and you have to kind of just work through them. There's really no way to balance, like, the speed at which you need to find product-market fit with, like, planning for scale that you don't have. In some ways, you have to just go and run into the wall and then figure out how to climb over it. And so I think, like, maybe a couple of years or 3 years ago, the outages were the thing that, like, we really tried to plan to not have. to a degree. But, you know, even things that we kind of anticipated or didn't— there are things that we didn't anticipate that happened. In the last year, I would say the thing that I wish was different, though I didn't plan an attack to prevent it, is that people still believe, largely informed by the LLMs, that we are still just a WebRTC transport company.

Hmm.

When we are very much like an AI infrastructure company.

Yeah.

And I don't mean because of the use cases. If you look at the product that LiveKit is today, the platform, it's not just like network transport. It's how do you build the things like these agents? How do you like test them and evaluate their performance? How do you deploy them? How do you like scale them up and give them access to the telephone system? Allow them to run inference super fast at the edge? And then, you know, all the way at the other end of the spectrum is like, how do you observe and monitor these agents too. And so it's like really this kind of end-to-end stack, but the LLMs still think that we're transport.

I'm curious just your guys' take, whether it be specifically to LiveKit or more in the ecosystem around like the LLM relationship for LiveKit versus the harness relationship, and how are you intersecting and playing in both of those worlds?

You could argue that we haven't stayed in our lane from at least when we started.

What is a lane?

What is a lane? Well, our lane actually, I believe, is like horizontal across the stack. But I think the thing that we are going to heavily resist doing, at least anytime soon, is going up the stack and competing with developers that build on us.

Yeah.

We don't want to do that. And going down the stack, and I view LLMs, foundational models, as down the stack. We build a harness that sits on top that wraps the models itself. And I think that the other thing about kind of the infrastructure we build versus an LLM is it's 2 very different skill sets. Training models, model research, the techniques to advance the capabilities of those models is fundamentally quite different from a technological expertise perspective as from distributed systems. We are like a distributed systems infrastructure company, not a model research company, though we do train models.

Yeah.

But they're very small models for, like, conversational dynamics. So we have a model that predicts when, you know, a person is done speaking so that the AI can start speaking. Or we have a model that allows the AI to predict whether the user talking to it is interrupting it or simply backchanneling, like saying, oh, okay, okay. And it's not a true interruption. So we train small conversational dynamics types of models, but not like those text-to-speech or LLM models. For those, we partner with folks or we leverage like open weights models. So we started to host Gemma in our own data centers now. We've started to host Deepgram speech-to-text all around the world in our own data centers.

Yeah.

And so we are hosting some of the models, but not going and trying to like build the model company competitor.

Sure.

I think, like, harness versus model. You know, Anthropic published this post about, like, how they built their harness, or this harness engineering post. And one of the things they talk about is, like, what should belong in the harness and what goes into the model. And something that they found is they put all these, like, things, this, like, kind of deterministic software into deterministic code. into the harness, and then as the model gets better, they have to keep removing these pieces.

Yeah.

There is a point, though, I believe— I've been thinking about this lately— of, like, harness irreducibility. So there is a certain point where you can no longer stuff the things outside of the model that are in the harness back into the model. Where I think that that split is, generally speaking, is there's 3. The first one is you can't put networking into the model.

Yeah.

So it's not gonna be doing any of the network infrastructure transport stuff that LiveKit does. It will manipulate or use that technology, but it will not have it within the weights. The second thing that it will not do, I believe, is memory. Well, I have other takes on this too.

Yeah.

Okay. What I don't think it will do in the short to medium term is memory.

Okay, fair. Let's go with that.

The reason why is that, at least for now, memory is bespoke. Everyone's memory is their own.

Absolutely.

Right? Every agent's memory is their own.

And for humans, by the way.

Yes, for humans, the human counterpart. You know, we all have our own memory. And so if you try to put that into the model, that model is actually shared. Those weights are shared across thousands of customers.

Right.

You're batching requests. You know, you're basically running a bunch of people's inference through the same model. And so if you put bespoke memory into the weights, then suddenly, like, I mean, that's a leakage of data privacy. But the second thing is, Suddenly, like, that model can no longer be multi-tenant.

Right.

It has to only be single-tenant. And that has an implication on the performance of how that model is delivered to people, right? Suddenly you have to have, like, dedicated GPUs to, like, a single customer. And so it's not the most economical thing. And thus, I think that memory for now will stay outside of the model and in the harness or in a separate auxiliary system that is manipulated by either the harness or the model. And then the third thing is there is a part of compute that is also outside of the model. And that's the stateful part of compute, right? The agent relies on the model to be a brain. But again, that model is multi-tenant. It has to be stateless. Because it's multi-tenant, it's stateless. And so where do you keep all of the context, right?

That's right.

You have to keep it somewhere that is bespoke and connected to the user that is interacting with it. And that is not shared. And so that is the one piece of compute that has to stay outside. And so I think that the harness eventually just gets reduced to 2 things, to 3 things. Well, prompt is part of the context. So it gets reduced to 2 things, context and tool manifest.

Okay.

That's where it goes.

And so then if that is true, and I like your framework there, and I'm not gonna be sophisticated enough to argue your framework. So let's hold this together. How then, when LiveKit is building your infrastructure primitives, kind of supporting the distinction of the voice agent? Because I think that's what's something so unique about what you're doing.

Yes.

And is really a part of the future of AI that is still emerging under our feet. I think you could make the argument that text, people feel familiar with that, they're comfortable with that. But voice, wow, that's maybe a new frontier.

Yes.

And I'm curious to hear the consumer reception from your opinion. But actually, first, I'd love to hear just kind of the idea of building agents with voice and, like, how that actually has unique needs and runtime and infrastructure, etc.

Yeah, so that— and I think, like, also this whole part about me talking about this irreducibility of the harness, it ties into kind of specifically, at least right now, specifically voice agents. I do think the future of agents is also going to look like this too. But there's a fundamental paradigm shift when you go from an interface that is chat To an interface that is speech.

Naturally.

Right? When you chat with another person, when you send them a text message, you don't expect them to respond right away. And so that's your prior for when you chat with a computer. You don't expect the computer to respond right away either. So it's very forgiving for latency. You can effectively take a request, you can send that request to the server, you can wait, you can have it compute, you can send a response, and then you sever the connection. You know, discrete computing that powers all of the web over HTTP is fine for chatbots. But your prior for talking to the computer is talking to a person. And for, you know, a couple decades, for me, 4 decades. For me, you know, we've all been conditioned that when you talk to a person, You're gonna get some kind of response in 250 milliseconds or less, whether that's you saying something or laughing or nodding your head or moving or saying something back to me or interrupting me when you disagree with me. That's supposed to happen really fast. And so you just cannot afford to send a request, wait, send a response, sever the connection, reconnect.

Timeout.

You just can't do that. You have to be persistently connected. It's continuous computing. You are constantly— the user is constantly speaking or streaming their audio, whether they're speaking or not. The AI is constantly reasoning, processing the speech, thinking, and then deciding whether to backchannel, deciding whether the user's done speaking and it can start speaking, deciding whether to call a tool. All of these things are continuously happening for as long as the session goes. We had this realization kind of semi, again, by accident when we were launching voice mode. OpenAI put a meeting on our calendar with no subject. And so we go into it and they're like, they're like, hey, we're launching voice mode. How do we scale voice mode? And I'm like, you're OpenAI. Why are you asking me? I don't understand. Aren't you guys like the best engineers?

Yeah, this is a thing.

And then they said, well, ChatGPT is a website. Voice mode is fundamentally different.

Yeah.

And so how do you guys scale these kinds of stateful applications that are like scaling Fortnite, not scaling a web app? And so—

Back to the video games.

There you go. That was the light bulb for us. We're like, okay, wait, actually there is this paradigm shift. When the interface shifts underneath, all of the infrastructure actually has to shift and go from stateless to stateful computing. And every piece has to be reimagined. Because there was nothing else in any of the adjacencies to test these things, to deploy them, scale them up, and observe them, we're like, well, we want developers to be able to build these things, so I guess we've got to build every piece. That is what led to LiveKit's kind of expansion into all of these different areas. But yes, it's fundamentally, at the end of the day, agents, whether those agents are So whether that agent is in a computer that you talk to or on your phone or a wearable, or whether that agent is like walking around with arms and legs and like can pick up objects and fold clothes, which is the thing that everyone's doing right now on X. These stateful computing is what unifies both of these kinds of new types of software.

So I think a few weeks ago, OpenAI released ChatGPT Live.

Yes, GPT Live. Yes.

Which is a duplex model and doing a lot of what you were sort of mentioning. of thinking while being able to talk. And it's really good at nodding and uh-huh. And if you interrupt it, it stops talking. Could you tell us a little bit around where you think gaps still exist in building this really natural experience where your grandma— I know they had a cute launch video about that— feels comfortable talking to AI?

I think we're already or OpenAI is already pretty close to that, right? The way that this new kind of model, like, can empathize and can express itself with a range very similar to, like, the range of human expression, that part's very magical and very compelling, I think, for casual conversations with an AI. There are still some gaps where I believe those gaps can be closed through just more training data. Like, one gap in particular, if you talk to it for a while, is that the expressiveness of it, the like, oh, okay, et cetera, and like the size and the laughter, et cetera, They can almost feel, like, fake because it does it too much.

Yeah.

Right?

Yeah.

And so I'm not sure what is actually influencing that, whether there's some kind of RLHF where they've nudged it in this direction. But I think more training data and maybe more kind of steering or human feedback can, like, get it to feel a bit more natural. I think that's an obvious gap. There's another category of voice AI, not like talking to your grandma. Well, maybe if your grandma's actually like calling into the doctor's office and trying to get on her doctor's calendar. That's where I think that there are more gaps with these like full duplex models than the kind of traditional approach that is pretty widespread today in the industry, which is the cascaded pipeline of speech-to-text, LLM, text-to-speech strung together. 3 different models. And I think the gap there is going to come down to instruction following and tool call reliability. Those are the 2 things, right? So instruction following, extremely important because you ultimately want— you need the model to do the right thing, especially in like I'm calling a a hospital and I need to get on my doctor's calendar.

Right.

I have some affliction I need advice on. You definitely need it to follow instructions, not hallucinate, not like randomly scream. That has happened with like older full duplex models, pre-GPT Live. Maybe it happens with GPT Live. I'm not sure. I haven't encountered it personally. But so that part is really important. There's another part of instruction following that's also a little bit underappreciated. Which is what a lot of companies do is they'll put guardrails outside in the harness external to the model itself. And those guardrails make assertions about the text that is going in or the text that is being generated. But when it's audio going directly in and audio being generated, it's not so easy to write an assertion around an analog signal. You have to convert it basically into text, which takes a bit more time. that can impact latency. And so then what you might say is you might be like, okay, well, I'm gonna stream the audio. And then if I get text back that violates my guardrails, then I'm somehow gonna like kind of try to wardrobe malfunction it and like cut it off, like once it might have already said the wrong thing. So that is still possible with these models. There needs to be more training data, more mileage put on these things before I think that they're robust enough that like the largest enterprise in the world will power like their healthcare use case. So, instruction following, then, yeah, of course, tools, you need to be called reliably because, you know, what if, like, the put the appointment on the doctor's calendar function doesn't get called? Well, then, you know, you just went through this whole process and then you actually don't end up having an appointment.

Today we've covered a lot of ground. We've talked about early 2000s. We're talking about, like, the invention of React.

It was, like, higher level and then now we're really deep.

We're talking about the bleeding edge of voice models and infrastructure. Russ, tell us, what are you excited about on LiveKit's roadmap this year?

Oh, boy. What can you share? What can I share? I know one thing I can share because we've already shared a lot about it, but I am super excited about physical AI. And a lot of people are too. But, you know, I think physical AI is something that I believe eventually will be even bigger than digital AI and agents that exist in our phone and our, you know, or that we interact with through our phone and our laptop. I think that, like, this entire world of being able to do things out in the physical world and help humans I can tell a short anecdote, like maybe the physically AI use case that I'm most excited about.

Yeah.

That if I wasn't working on LiveKit, I would go tackle is my wife's stepdad passed away late last year. 2 years before that, he ended up having early onset Alzheimer's. And so we moved him into a care home to work through that challenge. And it's insanely expensive.

Yeah.

Like, it— for something that isn't even fancy, was like between $15,000 and $20,000 a month. And it's just like, you know, he didn't have that much savings. And so it's just drawing against anything he had. And it was a really stressful time financially for how we were gonna make sure he had the help and support that he needed. And that's an area where I'm really excited about what robotics will be able to do and humanoids might be able to do, just because it won't just provide better economics for the home itself and him access to cheaper care. But I think also, he has children and a family.

Yeah.

And they have their own families. And so it can also be a burden as well on the children for how they're going to how they're going to deal with this challenge with so many things going on. It's not that there's a lack of desire to, it's just hard to load balance all of these things in life. And so I think it just helps so many people all around, you know, with a real human and real part of the arc of a human life. And so, yeah, I'm super excited about physical AI. We power a lot of the humanoids that you see already.

Wow. Oh, okay.

It's a use of our technology that, unlike almost everything that has happened to us in this journey with LiveKit, it's the one thing that was actually part of our seed pitch.

Oh, yeah.

Because when you're raising your seed round, that is still one of the phases of the company where it's, like, purely about the founders and the story. And I was like, okay, well, yeah, pandemic. I'm sure we'll get out of the pandemic at some point. So audio-video streaming with each other during the pandemic. Like, but what's like the real thing that's going to require like video and audio and stuff? Well, it's like, of course it's going to be robots. Like, you're going to talk to robots.

Totally.

They're going to see the world. Sight is video for a computer. And, you know, it's not— you can't— you're not going to walk up to its keyboard. You're going to like just talk to it because it's not even next to you. And so robotics has always been part of the kind of aspiration for what our infrastructure would be able to help with. So I'm really excited for that, and you're going to see us go a lot deeper into that space.

Well, we will certainly stay tuned.

Thank you.

And as we come up on time today, we'd love to ask you our hero closing question for the podcast. We ask this of all of our guests. Who is a woman who has had a profound impact on your life and your career?

Can I name 2 women?

Yeah, the more the merrier.

Okay. It would have to be my, my twin daughters. Oh, I love that. So they're 9 months old.

Congratulations.

It was a complete surprise. We were actually planning on never being parents. I had my baby, which is the company, and it's growing quickly. And we got a surprise from the universe, and we have these amazing twins, and they're the lights of my life. I don't want to cry on this podcast. No, but they're the lights of my life. And it's the thing that I never knew I was missing. It's incredible. I know Jia's like 6 kids or something like that. Yeah, that's exciting. He texted me when he heard the news, and he was just like really, really happy for me. But, and I think he's happy when people are just having kids in general. But no, I think it put a lot of like what the work that we're doing here into perspective.

Absolutely.

I think the other thing is that business can sometimes test people on their values.

Yeah.

Sometimes there's temptation that you might optimize for success or winning over your values. I think everybody struggles with that. And especially founders probably struggle with it. But the thing about children for me, the reason why I say it's such a profound effect on my life is it kind of always centers me around my values, not just because I judge myself, but now I think about how my children will judge me.

Yeah.

And the kind of person I am to other people in the world.

That's right.

And what kind of example I wanna set. And so, yeah, they're just like, yeah. the deepest impact of anything I've ever done in my whole life.

Well, thank you for sharing that.

Yeah, thanks for the question.

Of course. Thank you for joining us today. It's been such a pleasure to have you in The Room, a nice room today.

Yeah, absolutely.

Super fun. We're so excited for all that's to come for you.

Thanks so much.

Thank you. And we believe 2 is better than one. We, we agree.

Yeah, exactly. Yes.

We're a pair.

Yeah, yeah.

Awesome.

Love that.

Thank you so much for joining us at The Room Podcast. If you want more from The Room every week, subscribe to our newsletter at theroompodcast.com/newsletter. We'll be back next week with a new episode and inspirational guest. Tuesday, 10:00 AM Eastern, 7:00 AM Pacific. See you in The Room.

And now a message from one of our sponsors, Rippling. One thing we constantly hear on The Room is that great operators don't just need more data, they need a faster way to turn that data into decisions. As a founder, I know how painful it is to wait days or even weeks for a report while a real business problem is unfolding. That's what makes Rippling AI different. You can ask Rippling AI, is our sales team at risk of missing quota? And Rippling AI instantly pulls real-time data from Rippling and your CRM into presentation-ready dashboard data showing quota attainment, headcount trajectory, and monthly revenue to quota by region. But it doesn't stop at the insight. Rippling AI can act on what it sees, Recommending and building an automated recruiting workflow when open roles and slow time to fill are putting quota at risk, all while following your existing permissions and approval processes. Question answered, action taken, problem solved. Ready to rule your business? Head to rippling.ai/theroom and sign up for exclusive access.

And now a word from our new sponsor, Gradiel. Whether you're building a company, launching a product, or growing a brand, you know the idea is just the beginning. And marketing's bottleneck has never been creativity. It was always the work it takes to get an idea out the door. Gradiel handles the operations so that marketers can stay on craft. An agent will learn your process, work across the systems you already use, and delete point solutions you don't need. Designers will ship on-brand pages without counting prompts or tokens. Product marketers will launch across markets in days, not weeks. And digital experience teams test customer journeys, catch what's not working, and fix it All before launch. Gradio learns about your brand, business, and process and brings that continuous context to every workflow. Every task gets done your way with brand, quality, and approvals built in. No more token maxing, slop, or runaway AI bills. More time to think bigger and create the impact people remember. Gradio, the system of work for marketing. Ashurst Perkins Kewey is a leading full-service international law firm known for providing high-value strategic solutions for startups and investors. Lawyers at Ashurst Perkins Coie support the most innovative entrepreneurs and investors in fast-moving and high-growth sectors across the country while addressing the myriad of legal needs that arise when fundraising and scaling their businesses. The firm's tailored legal services are informed by understanding of each of the client's discrete needs. To learn more about Ashurst Perkins Coie, visit their website, ashurstperkinscoie.com.