Tavus: The AI Human Platform

I'm excited today to welcome the founders of Tavis, to Quinn and Hassaan, who just raised a $40 million Series B led by CRV. Tell us what Tavis does.

We build AI humans. We're an AI research lab that focuses on teaching machines the art of how to be human. So teaching them to see, hear, respond, act. Even look like humans do.

What does that look like? Can we take a look at a demo?

Yeah, that'd be great. Here we're about to talk with Charlie, one of our awesome Tavis pals. And it's really cool because you can video call them, you can audio call them, you can text them even. They're always on, they're thinking about you, they're agentic.

They look real, like a human.

Yeah, absolutely. And they can see you, they can react to all your expressions, your emotions. They really feel like you're talking to a coworker or a friend rather than a machine.

And I think the impressive stuff is that this is all running in real time.

Yeah, absolutely. It's all running in real time.

Let's actually see a very short clip of it interacting.

Hey, what's up, Dom?

Good morning, Hassaan. How may I assist you today?

Yeah, can you tell me what my schedule is like for today?

Certainly, sir. You are currently scheduled for a YC recording of some sort, followed by a 2 PM product meeting and a 3:30 investor call.

Great. My recording's running over. Can you draft a quick email saying I'll be 15 minutes late to the product meeting?

Right away, sir. If you check your email, I've already prepared a message informing them of your delay.

Beautiful, thanks.

My pleasure, sir. Can I be of any other assistance today?

Very cool, thank you. That was a cool demo. So tell us a bit about who are some of your top customers.

We have a pretty awesome diverse set of customers, everything from like small startups to like Fortune 10 companies, some of which are like Amazon, Better.com, Alibaba. That are using our interfaces and our models to build these really cool AI employees.

What do these AI employees do? I think you mentioned you have 3 categories of types of applications.

Yeah, so there's 3 big buckets that we really do a lot of work with. The first one is learning and development, so a lot of training, education, things like that. The second one is healthcare, so think patient intake, nutrition coaches, elderly companionship. And then the third bucket is go-to-market, so anything from an AI SDR to an AI solutions engineer to a customer support manager. So it's really across the board, and I think that's been part part of the fun of building it too.

It's amazing to see all of the really awesome AI human use cases that we're seeing people create, like just the craziest thing that we would never have imagined.

So it's basically a virtual human that's doing all these tasks over video, right?

Yeah, I mean, it's, you know, it's an AI human on video, but with the new updates also you can video call, you can audio call, you can text you. It feels like a coworker or a friend.

For that to be the case, it needs to actually feel human. It needs to look like you, sound like you, understand, interact just like a human would. That's where it really gets its special touch.

There's a lot of custom use cases. The way you guys work right now, you got to here with being an SDK where customers like Amazon, Alibaba take your SDK and build the custom AI employee, right?

Yeah.

But that was not the case all the time, right? Yeah. I think when you guys went through the batch, back in summer '21. This is way before the ChatGPT era. You were one of the OGs in AI before AI was a thing. Tell us about that era.

We like to say that we were doing AI before it was the cool thing to do back in 2020 and 2021. I mean, when we were with YC, there was not that much that you could do with the models of the time, right? And so we were sort of limited by what we could create. And so at the time, the best that we could do with these generative models was do what we call like lip sync infill, which is like you could, and we use it for AI personalized video. So basically you could record one video saying, hey Diana, saw you worked at YC, and then you could send out thousands that were all personalized, lip synced with your voice that would say, hey there, Alex, saw you worked at X, Y, ABC, right? That was what you could do at the time. You could create personalized video 'cause you could take a segment and really scale it up. Over time though, of course, the models have evolved and our research has evolved to be able to do a lot more.

You guys stayed the course in this direction because before you were selling this tool to sales team and then evolved into the SDK. So tell us, what was that pivotal moment?

I mean, I think it was a few things. I think it was really understanding who we wanted to be as a company, right? I think that after we did our Series A, there was this crucible moment where we're like, hey, are we going to be an AI sales company? Or are we going to be focusing on developing these models for like human computing, essentially, like to develop the next generation models for like human behavioral analysis and simulation. And we decided that, hey, like we don't want to go down a path of being an AI sales company. That's not just not the DNA of our team. It's not who we are. And ultimately we didn't really think that we personally could make a massive business with that. And so we decided to take the path of saying, hey, we're actually gonna take a step back and we're gonna go and focus on building the models and serving these as a next-generation interface and as an API that you can consume and build on.

I think a lot of it came back to our core too. I think we went through the YC batch and we were pushing, we were trying to get as much momentum as possible and build, build, build. We got a ton of customers that came to us looking for very specific things, which was awesome at the time. Then at the end of the summer, we took that step back and said, wait a sec, these are really cool customers to work with, but is this really who we are and what we want to be building? Took that really critical reflection and ultimately made that jump into the API and SDK side.

The impressive thing is having the courage to take that leap because you essentially, I remember having this conversation, you churn all those customers and you were on the track to get Series A traction before all that. You were growing quite quickly and to say that is not the path and our DNA are more extremely technical, you have some really impressive research work that you've done. It was not to be a sales company and to be now what you are becoming, an AI research lab, right? Took a lot and it was the right choice, evidently, because now you are building foundation models, not just for rendering, which is what it was before, and now also perception. Can you tell us about these two sides of the yin and yang?

I mean, ultimately came down to like we saw a larger vision that we could really— we wanted to pursue. And I think a lot of the team has had this vision for a long time around machines that meet us where we are. Instead of us having to learn how machine talks, they can meet us and they can video call us, phone us, text us. In order to do that, in order to teach machines the art of what this is, because it's like a dance, then just giving them the face isn't good enough. Also, the face won't be good enough ever unless you teach a machine to see the way that we see and teach them perception, contextual perception. It was both necessity to make the rendering models better, but also part of if if you want to build these AI humans, then they have to be able to see our expressions, our gestures, because we speak as much through our face and what we don't say as we do through our words. And so we really focused a lot on trying to collect all these signals that otherwise weren't collected before and then make meaning out of them. So teach the machines the relationship between what you said and how you said it and the expression on your face and like how that was a reaction to what I had said, right? Like all those things became essential to building these AI humans.

And I think a key thing is that this is only possible to be built now because for this to work, it needs the real low latency to have a fast response time. I mean, the human perception is more than, what is it, 10 milliseconds?

Yeah, yeah. Like, you know, a great response back and forth happens in like less than 200 milliseconds.

So you guys have some new product launches that are coming up. Can you tell us more about it?

I mean, yeah, so for the past couple of years, we've been working on the foundational models to teach machines to see, hear, respond, even look like humans do. And with this new product launch, we're actually essentially bringing AI humans to life for regular consumers and prosumers. And we really think it's a transformative moment where we feel like we've been in the command line era of AI. We're just like in early computing where we went from these command line computers to GUIs. That allowed millions of people to use machines. I think with what we're calling the Tavis pals, it's going to be the same thing where you'll have this AI human that meets you where you are. It can video call you, it can text you, it can, you know, you can call it. It's proactive, it's multimodal, it's agentic, so it can go and do things for you. And it feels really, really natural because it has a high degree of emotional intelligence powered by these really amazing state-of-the-art perception models.

That's pretty cool. This is a big evolution. You're going from serving very technical users with the SDK that need to be good software engineers to ship an AI human, now to someone who's not technical to be able to just launch it. And that's going to unlock a lot more interactions. I mean, if you guys succeed with this, you're going to probably be doing at some point 100 billion Tavis interactions, right?

Yeah.

How will the world change when you get to when you get there?

Yeah, I mean, I think our team will cry happy and sad tears. But I think what we're really excited about is solving the human computing problem, which if you solve, then essentially using a computer becomes something that's second nature. You don't have to learn how to do it. It just feels like talking to a friend or coworker. So you'll talk to your AI doctor or your AI therapist or your AI assistant sidekick. Everyone will have Cortana and Jarvis sci-fi dreamed of in their hands. It'll understand you immensely, and that's a future we're really excited about.

It's the start. It's the tip of the iceberg. These are the 5 first-class citizens showing what AI humans can and should be. Then from there, it will continue to expand where people can create their own and build on top of those.

That's a very cool future where Tavis is going. This is the other side of it where there might be a bit of concerns from people in terms of, alignment and a lot of these jobs that AI are going to start taking. How do you guys think about that?

I would say that, like, certainly some of that will happen. I think at Tavis, like, we, we don't try to shy away from that and say, oh, we definitely, definitely won't be replacing jobs. But I think that the key goal is not to replace humans, but to replace bad machines that have been already put in place and are causing regression. Like, telehealth already exists. You already don't get to talk to a front desk receptionist or a nurse, and it's just the worst experience for everyone involved. If you have an AI intake assistant that really understands you, it sees you, it sees how you're feeling, and it spends time with you in your language, like, that's a better experience. I also think that it's a little bit of a privileged approach to be able to say, hey, like, if you had asked me like 2 years ago about AI therapy, I'd have been like, that's crazy, it's a bad idea. The reality is most people in the world can't afford therapy. They don't have access to it. The alternative isn't a human, it's nothing. If we can deliver an 80-90% good experience, it's not actually replacing a human, it's putting this AI human somewhere where there was no alternative. That's something that we find to be really special.

How do you actually build these AI humans that have empathy?

That's a good question. It's interesting because Human conversation is incredibly nuanced. We like to say at Tavis that human conversation is an art. It's a dance. Like, this is a waltz that we're doing right now. And machines are in the corner doing the robot, and we're trying to do a waltz. And so to bring machines to be able to do the waltz, the first thing is you have to give them the right signal and be able to collect the right data. And so that's why we spent some time on perception, being able to collect the most nuanced data on that eyebrow twitch you had or the slight smile smile, to be able to collect that data and then to be able to actually form understanding around it. And the understanding piece and the relationship piece is actually really important. So teaching them the relationship between the eyebrow twitch and what was said before that, like what was that in response to. We spend a lot of time modeling those things. And essentially you can summarize what we're doing as creating human simulation models. We're simulating human conversations and reactions and expressions. It's a lot of fun and it's a very human thing to do, which is always a little tough for research teams. There's the research angle, it's very, very human rooted.

A big part of what's important here too is how we bring it to the world. I think one of the lessons we learned really early on was we need an amazing demo that shows someone the magic, the experience because we've adjusted to how we talk to machines for so long. We type instead of talk. We have all of these lossy mechanisms for how we change information and We're almost having to reteach humans how they can communicate with machines, how they can actually talk naturally and normally. It's not just the models, but it's actually creating the experience around it where people are able to have this really refreshing, this new experience of talking to a machine.

Yeah, 100%.

What advice would you have for founders that are just getting started?

Have more conviction in yourself and the vision. Whenever you're a young founder, you can be more easily swayed by opinions and feeling like you need to do you need to follow a certain path. And I think it took us a bit to figure out what we believed in is what we should be working on.

In particular, was that crucible moment? Because I remember that was a pretty big pivot for your company, and it took a bit of time for you to come to your own belief.

Yeah.

Because you had a bunch of other people telling you otherwise, right?

Yes, absolutely.

And it was a struggle.

Yeah, it was. It felt like you might let someone else down, and ultimately, is a company that you're building and you have to have full belief that the work that you're doing is a form of love. If you don't have that, then it's not gonna be successful no matter what.

It's a craft. I think that was probably actually one of the most difficult times at Tavis looking back at it when we let those opinions influence us, right? I think it impacted who we hired, what we were building, who we worked with. Like it went throughout the business. It wasn't even just about the direction.

Yeah.

One of the learnings was definitely about not even believe in ourselves, but just have a deep conviction in what we're doing and let that flow everywhere throughout the business. But the other one is the only thing that matters is momentum. If things are not moving, then the business isn't moving. And it's in the small things, it's in the big things. But the only thing that matters is every day something needs to happen. And I think driving that through and pushing that through is one of the few things that has kept things moving forward day over day.

100%.

I think this is a good way to end. I think we talked about in one of the videos recently that the most startups really only have is just speed. And you guys have done it.

Our take on it is we are 6 months ahead and we need to keep moving as fast as we can to keep that up.

We have our core motto at Modo at Tavis is, "Faster, faster, until the thrill of speed overcomes the fear of death." Okay. And it's worked really well for us so far.

Faster, faster.

Faster, faster. All right, guys, congratulations.

Thank you.

Thank you for having us. Thank you for coming.