a16z on AI Voices: Call Centers, Coaches, and Companions with Olivia Moore & Anish Acharya

Hello, and welcome back to The Cognitive Revolution. Today, I'm speaking with Olivia Moore and Anisha Charya, partners at Andreessen Horowitz and fellow AI scouts, who are constantly tracking emerging technologies and consumer behaviors, and who in recent months have really distinguished themselves as keen observers and eager early adopters of AI voice platforms and products. In this conversation, we review recent developments in voice AI technology and also explore how voice AI is already starting to transform business operations, user experiences, and human-computer interaction more broadly. On the technology side, we discuss how recent multimodal models have simplified the model stack, and with it, the application development process. How reduced latency and improved interruptibility are enabling far more natural conversations than ever before. And how products including past guest Hume AI's Octave model, Google's interactive version of NotebookLM, and the very viral and incredibly natural-sounding Sesame are all beginning to demonstrate a remarkable level of emotional intelligence. Both in terms of their ability to understand the user and the expressiveness with which they can communicate back. On the impact side, we cover a bunch of really interesting real-world applications and trends, including the company Happy Robot, which uses voice AI to handle complex negotiations and build rapport with truckers in the context of freight brokerage, vertical solutions for restaurants and other types of SMBs, and how business owners are often employing these systems to handle after-hours and other calls that they can't answer themselves. How enterprises are using the technology to facilitate meetings and provide real-time coaching. Why we haven't yet seen a major impact on call center jobs and how soon that might happen. Why Apple is now saying that Siri won't get a major update until 2027, even as so many other things are clearly starting to work. And of course, the ongoing rise of AI companions for kids, for seniors, and for the lonely. Along the way, we also touch on a couple philosophical questions relating to human labor displacement and the delicate balance between protecting consumers and encouraging innovation. As someone who's been a career entrepreneur and genuinely loves using OpenAI's advanced voice mode as, among other things, a biology tutor and a real-time video game guide for me and my kids, I am super excited about this technology, but also legitimately concerned about just how scammy the world could quickly become if the models get even just a little bit more lifelike. It had been a while since my last voice AI red teaming exercise, But with this conversation in mind, I did go back to 2 reasonably well-known AI calling agent platforms to which I had previously reported flagrant vulnerabilities and found that sadly, a year later, both still allow me to quickly clone Donald Trump's voice and scalably call anyone and say anything in that voice, all with no meaningful controls. This to me does strongly suggest that we need new rules for these sorts of products sooner rather than later. Both to protect the public and to protect the AI industry from its most careless developers. As it turned out, Olivia offered a brilliant twist on my do-not-clone registry idea, which could both expand economic opportunity and protect the public from AI personation, and which I feel it is truly time to build. As always, if you're finding value in the show, we'd appreciate it if you'd share it with friends, write a review on Apple Podcasts or Spotify, or leave a comment on YouTube. We welcome your feedback too, either via our website, cognitiverevolution.ai, or by DMing me on your favorite social network. For now, I hope you enjoy this exploration of the rapidly evolving world of AI voice interactions with Olivia Moore and Anisha Charya, consumer technology investors, AI scouts, and partners at Andreessen Horowitz.

Olivia Moore and Anisha Charya from a16z, here to talk about the future of AI voice interactions. Welcome to The Cognitive Revolution.

Awesome. Thanks for having us.

I'm excited.

So let's see. First thing I wanted to say was just, you guys are part of a group that I affectionately refer to as AI scouts, people who are out there on the edges of what exists and kind of exploring it in, I think, a lot of different ways. So before we get into the actual object-level stuff of what you've found and what you think is coming in the realm of voice, I'd love to just get a little bit of like maybe meta lessons or, you know, specific alpha tips for how you do such a good job of this. Like, where do you go for information? What's the sort of top of funnel for you? How do you know you're onto something? I think people could learn a lot from your example in that respect.

Awesome. Yeah. I mean, in many ways, it's our job to be kind of chronically online and tracking every new thing that happens, especially as consumer investors. So, It's something that we've tried to hone over the last few years in particular. It's so interesting because there's a pretty big delta, I think, between what AI scouts or AI experts, early adopters, where they're spending their time and where they're talking, and then where kind of normal consumers are. So, we try to be in both places. I would say, like, Twitter, of course, is where most founders, AI founders, are announcing new companies or new models, new breakthroughs. AI newsletters have been massive, meetups. But then in terms of like where real people are sharing what they do with AI, it's mostly on places like Instagram, TikTok. YouTube is a shocking one. YouTube is actually like the number 1 mobile app and the number 2 website in the world. And so we actually find that for many companies in the consumer prosumer space, if you look at traffic, like by far their number 1 referral source from social is YouTube. Like, there's this whole separate economy of kind of YouTube influencers or YouTube creators who are making like how-to content about using different AI tools. So I would say we try to track all those places. In terms of what's an early signal to us, like, often we'll see normal people, again, usually teenage girls, to be frank, like, trying to manipulate ChatGPT into doing something, like to be a therapist, to be a friend, to be a coach. And once we see something like that, it's like, okay, the consumer pull is strong enough that probably there can and will be a couple standalone, more focused products here.

I think the other thing, Nathan, we try to do a lot of is just use the products, you know, and that sounds obvious, but it's surprising how few people seem to have actually tried like Operator, Deep Research, DeepSeek, O1 Pro. I mean, Kria, you know, these are not super obscure long tail products. And consumers find their way to these products, but for all the folks that are sort of insiders and are paid to be doing this work, that, you know, still surprisingly few actually look at those. It's just a great way to build your intuition.

Yeah, that's my number one advice always too, is just get hands-on. Like, you can't really go wrong with that. So the one interesting thing there was it sounds like you're looking as much or even maybe more for demand-side pull as you are at the, like, You know, people, because people are all the time coming forward with their technologies to offer, but you're looking for people that are specifically trying to meet a need that maybe nobody's met yet and figure out what that implies.

Yeah. I mean, consumer is so random and magical that we try to let the data tell us, like, you can have the most tenured pedigree team in the world building a consumer app. And if it doesn't hit, it doesn't hit. And that can be for like a variety of different reasons. Like maybe it's bad market timing, maybe They got the product insight wrong, but some specific feature, or the product insight right, but some specific feature wrong. And then, you know, no one completes the onboarding. And so, I would say we try to let the data, in terms of what people are actually using, tell the story to us. And sometimes looking at the data of things like people pulling ChatGPT to these off-label use cases will kind of give us, like, a little warning signal or a heads-up that, like, okay, this is something, this is a behavior that's working. And so, we should— Keep an eye out for products that are kind of targeting this behavior.

You know, a great example of this, there's this old joke in pre-AI that like every social app would like collapse into being a dating app. And I think in the same way that every like large language model is, you know, being tortured into being a therapist so that, you know, it's like a funny thing to say at a dinner party, but it's also a leading indicator of what consumers want these models to do and some of the things that, you know, we want to see in the future.

Cool. Well, let's talk about voice. I would love to maybe just for starters, get sort of the tip of the top highlights. Like what products and experiences have you seen that are just the absolute best user experiences that are out there today? Hopefully I've tried them, uh, but we're about to find out.

Yeah. Well, maybe I can frame it and then Olivia can talk about some of the specific products. So I think the thing to resume all the way out is. Grounding ourselves in the fact that voice intermediates every human interaction and relationship largely, right? Like here we are obviously having voice intermediate our relationship and our conversation and the way that we get to know each other. So it really is the original most important form of communication, human communication, but it's just been completely unaddressable by technology because we've never had the infrastructure. So it's very interesting because so many of the other substrates that we're applying AI to are areas where we've also, we've had like a lot of historical technology exploration, whereas voice is just a complete blank piece of paper. And that's why I think we're as excited about the product implications as we are about the distribution implications of this sort of technology surface.

Totally. Yeah, I would say there's been a couple of surprising things to us in terms of like where voice is working now, at least on the startup side. A lot of the startups that are getting real traction in terms of net new companies and products are actually more B2B oriented just because there's so many businesses that are now running off of call centers or paying for 1 or 2 or 3 people, even for small businesses to answer the phone all day. And so once you're at a point where voice models can be anywhere in the realm of human performance there, like it kind of makes all the sense in the world to at least have the voice agent be doing your after-hours calls or your calls that would go to voicemail. So my guess would be that actually a lot of people have maybe interacted with an AI voice agent and not quite known it because it's been a business calling them, or it's been the receptionist when they've called to schedule an appointment or something like that. On the consumer side, it's been so far maybe a little different than we expected. I think most consumers have interacted with AI voice through something like a ChatGPT or a Grok, which are incredible voice experiences. More recently, something like a Sesame was a massive breakthrough, and that is still just Like a web demo, early version of what's to come there. And so my guess is when we see— the Sesame team is open sourcing the model. When we see models like that spread and become more accessible for app builders to build on top of, we'll see maybe a corresponding explosion in consumer-focused voice-first tools.

I mean, a crazy thing that happened, and things are moving so quickly, I think it's easy to forget these things, is 1-800-CHAT-GPT. Like, what was that? You know, and, and I think it's sort of whether it failed or succeeded, I think it pointed to an important insight, which is that maybe the first way most people in the world will actually experience AI is via voice, both as consumers and, you know, consumers consuming sort of business offerings.

Yeah. I think of my dear mama all the time who is now in her early 90s and lives alone and is Sharp, but not like an early adopter of new technologies. And for her, I think it's going to be Alexa Plus that is going to be the sort of big transition from— she already sits there and asks that thing to play music for her, but will she engage it in conversation? Just how natural will she find it? I don't know. But it's clearly going to be for her that form factor that could unlock a whole new set of things.

And what's very funny is, ironically, Mama perhaps is not exploring new technology, but also isn't that familiar with old technology, which is maybe that's, you know, she's calling you to get tech support and help using her existing products and devices. Like, I think the potential applications for voice as applied to seniors are super interesting. We've discussed it a lot. And it's not just access to the new things. It's access to the old things as well that they just never developed the skills to interact with.

Yeah, funny enough, one of my first GPT-4 tests going way back to the red team days was tech support for seniors. And the prompt that I found to work really well, which is exactly what I say to her when this, you know, exactly what you're describing happens when she calls me and says, you know, my friend, you know, emailed me and I can't find it. I always tell her, read everything on the screen from the top to the bottom. And she'll literally go like, okay, Verizon, you know, the time. And then eventually we get down to like, well, you know, where the issue is. And that same thing, you know, basically worked out of the box with GPT-4. And of course, that was text only at that time. But I do see that as a huge unlock for all sorts of different—

That's amazing.

—that she kind of struggles to access right now. If they can figure out the TV remote, then we'll really be in business.

No, exactly. Well, and I think most people don't have, like, an infinitely patient person like you to kind of walk them through how to do that. And so, I mean, even we saw recently, I think it was late last year in December, Google released kind of the Gemini models that could see what was on your screen and interact with you in real time. And it feels like we're right on the brink of models like that. OpenAI has one as well, becoming kind of API available and becoming ready for builders to actually capitalize on. And so once we see something like that kind of become actually usable, it's going to be massive.

It's also so interesting because it sort of points at something maybe Google and other search players should have done pre-AI. Which is how do you take everything on the internet and apply it? Like, the most important context is the context that's around me in my physical space. So the idea of being able to point your phone at the remote in the case of Nanob and be able to sort of debug the problem that way instead of trying to translate what you're seeing in the physical world line by line to either, you know, Nathan or to Google, it just like that interaction pattern doesn't make sense.

Hey, We'll continue our interview in a moment after a word from our sponsors. So one of the things I noticed in your presentation about this is that you said it's like basically solved, I think was the phrase. And I'm kind of wondering what you see as remaining, if anything. The Sesame model might have even come out since that presentation and certainly takes another step forward in terms of just Overall natural sound of the voice. I find the interruption mechanic still a little bit gnarly, and especially if there's kind of a multi-way conversation. Sometimes I try to demo advanced voice mode for people just to try to bring them up to— my goal is to make it a quick way to bring people up to speed on, like, this is where AI is at now, if you haven't been paying attention. But then those demos often kind of go a little bit sideways on me because the interruption mechanic is still a little weird. And if I'm doing it one-on-one, it's okay. I I kind of know how to use it, and it seems to be optimised for that one-on-one, but in the group setting, it doesn't do super well in many cases. Anyway, that's just me kind of identifying my remaining pain points. But what do you think are the sort of hardest things to get right or the most important things that still need to get solved?

Yeah, I think the things that have gotten— maybe solved is too strong a word, but very close to solved over the past year would be things like kind of basic latency and understandability. Which is like the difference between being able to have a conversation and not have a conversation. And so in most cases, I think the models are getting those right now. Latency is like less than half a second on most of the models, which feels very human-like. I think the realization that a lot of people had that have tried the Sesame demo is beyond latency, there's so many kind of speech pattern nuances that an actual human will say, which might actually sound like an error to the model. Like there are extra pauses, they're saying like, um, or hmm, or vocal inflection. And when you add those things like that in, which the Sesame team has done, it goes from being a voice that sounds like better, much better than an Alexa and Siri, but is still kind of robotic in some ways or still clearly AI into something that could be mistaken for a human. I think the remaining things for me, there's still a lot to do around emotionality. I talk to a lot of founders building voice agents who want the models to be able to understand what they're saying and vary the tone and the inflection based on that. So if the AI voice agent is going to say something happy or exciting, the voice should reflect that. If they're going to say something sad, the voice should be kind of like lower in tone and pitch and a little bit slower. And so that's something that we still need to solve. And then interruptibility is huge. I kind of think of it as like humans have also not solved interruptibility in conversations. Like, we still have the issue where 2 people start talking at once and you have to be like, No, no, you go ahead. And so we need a clever way for voice AI to be able to solve that in a way that humans maybe have not yet.

You know, I think this is why we've gotten where we need to get to for voice models to work. But I think what Sesame showed us is that conversation models may be very different from voice models, or maybe an extension of voice models. So even how are the 3 of us sort of coordinating on who's going to talk next? There's so much, you know, nonverbal communication that's happening, whether it's video or even in audio. Um, and, and the models have been trained really well to do speech-to-text, text-to-speech. Interestingly, less of the companies we're seeing are using native sort of voice-to-voice-to-voice models. So that's one opportunity. And then just generally understanding, as Olivia noted, some of the nuances of conversation and having that be natively programmed in. For example, when I start talking, I'm not exactly sure what I'm going to say. You know, it sort of comes like And that's true for all of us, whereas the AI knows exactly what it's going to say, which can sometimes be a little bit creepy. You know, it doesn't quite get out of the uncanny valley.

Yeah, there's a recent paper from Meta that you're bringing to mind where they're doing brain reading and have sort of established now in, you know, actual signal understanding terms, the kind of phrase level formation that happens like 2 seconds before you actually want to choose a stank token and then how it kind of gets down to like the literal next syllable that's just like a fraction of a second before you spit it out. And yeah, there's something— it does feel like maybe some of these things might be better served in some ways by a diffusion structure, if you could kind of go from coarse to fine, as opposed to always being one token at a time at the end. But anyway, that's more speculation. You mentioned the stack.

Actually, Nathan, can I add one more thing? The only other thing I'd add is that you actually do see varying levels of performance and sort of conversation quality, which is a very big predictor of driving business outcomes. So for example, we're investors in a company called Happy Robot, which is a voice AI for freight brokers. And if you just look at the quality of their text-to-speech and just how the conversation feels, it just feels way better than many of the off-the-shelf things. And this is because the team is more technical and has done more work under the hood. So yes, there's a bunch of other competitors who can provide a kind of You know, a fine commodity voice experience. But if you actually are a little bit more specialized in the technology, you can do something that feels more human, and that gives you permission to move into higher value conversations. You know, persuasion, negotiation, disagreement, like those are pretty nuanced conversations. And for the business to trust you with those conversations, you've gotta have a voice model that not just says the right things, but says them in a way that feels compelling.

Yeah. Negotiation in particular is a lot to ask somebody to delegate to an AI. Is this company actually doing that for the freight companies? They're doing actual negotiations?

Negotiates, befriends, disagrees. We'll send you the demo. It's, it's amazing. You should embed it. It's like, yeah, it's a real aha moment, I think, in terms of what's possible.

It's really interesting because I think to the point of like the LLM always knows exactly what it Could or should say. Like, there is a version of an AI voice agent that does negotiation that would just respond to the human and say, no, this is my best price. Like, this is what I'm going to offer you, and just kind of say it over and over. But if you launch that kind of experience to a user, like, they're going to try to then circumvent, talk to a human agent. Like, it's not going to— it's not going to feel like an actual negotiation to them where, like, they've given it their best and they've gotten a concession from the other side. And so, what Happy Robot has done, which is really smart, is they actually introduce extra latency by saying, okay, the voice agent will say, hold on, let me go talk to my supervisor, and put them on hold for, like, 5 seconds. And then, they come back with, like, a slightly better price. And of course, the voice agent knows, here is my actual max, this is how much I can kind of go up or down or move the price for the end customer. But they found that, like, the maybe acceptance rate of that kind of final offer is, I think, much higher in cases where the human feels like they've gone through an actual negotiation. Because the voice agent has kind of simulated that situation that feels like satisfied to them.

Yeah, I don't know how to feel about that, to be honest. I mean, it's genius and it's a little bit far out. What I guess, staying on this for a second, does this— do people know that they're talking to an AI when they're talking to this? It just— does it say that upfront or how do they—

Yeah.

Yeah, it discloses it. And what's actually most surprising is people don't, and you know, look, these are not, they're talking to, you know, truckers driving all over the countries in their big rigs. So these aren't exactly, you know, Stanford technology enthusiasts and they, they don't mind at all. Because again, I think that there's kind of our, our reptilian brain is so trained to react to these interactions in a specific way that once you get into the conversation, even if you intellectually know that it's an AI, you fall into kind of the rhythm, the expectations, the patterns. the cadence of the human conversation very quickly.

Yeah. It reminds me of something actually Anish says a lot, which is like, in the best cases, AI can be more human than the humans. The Happy Robot example is a good one where every time you call in and you reach the voice agent, he or she is— you get the same person every time. They're friendly. They'll listen to you, you know, talk about your day or what happened. They'll be very patient. They'll be sympathetic. Like, they'll spend all the time in the world with you on the phone if you want to. And so, It's actually in many cases, like assuming the voice agent can answer your question, which, which they almost all can now, like it's actually a better experience for the end consumer than, than if you get what can sometimes be a grumpy actual human being on the other end of the phone line.

Yeah. Superhuman patience is, turns out not that hard to achieve and quite valuable. And yeah, low, I mean, low to no wait times is also like a huge Yeah. Driving force for value. So I definitely see it. I've been really excited about voice for a long time, even though it's probably going to put me as a podcaster out of work before too long as well. You mentioned Siri a minute ago, and obviously they've recently made headlines in a seemingly negative way by saying they're not going to have an update until 2027, which just feels like possibly the other side of the singularity from where I'm sitting. So I don't know if you can make any sense of that. I guess another way to maybe come at that is like, how reliable do these things need to be? I mean, I think there's often this sort of, in AI in general, there's this faux comparison or sort of imagined perfection that people compare an AI solution to. And I always try to remind people, and Ethan Maugham, I think, has a great phrase for this also. It's like the best available or the best hireable human. for the job is really the, the comparison that you should be making. So is that what Apple is getting wrong here? Or like, how do you make sense of that development?

I have so many thoughts on this. I mean, I, one, I think it's for any consumer that interacts with AI products and also uses things like Siri, it's like a stick in the eye 5 times a day because Siri is still so bad at just the most basic. And then, you know, juxtaposed on all the advertising that Apple's doing about Apple Intelligence. It's not awesome. I think it really degrades consumer trust, you know, one. And then two, look, I think that AI does best when it is, you know, exploring the surface of human interaction, which is a little messy. And these large incumbent corporations are designed to take the humanity out of every technology product. So there's this sort of almost irreconcilable tension between the two. So, and look, the more they try to neuter the AI, the more dissatisfied consumers are with it. Some of the Genmoji stuff, like it's a valiant effort, but it just looks terrible. You know, I don't know. Maybe some people think it looks good. I don't. So I just, I think it's going to be a very difficult spiritual problem for these companies to resolve because just, you know, between the committees and the lawyers and the whole posture that large incumbents have, it's going to be hard for them to embrace the messiness of AI. I don't know. What do you think, Olivia?

Yeah. I mean, I think we saw the reaction to the AI generated text and notification summaries on iPhones. And I would guess that kind of spooked Apple a little bit. Because for Apple to launch a new AI product, to Anish's point, it has to be production ready to land on hundreds of millions of iPhones of people of all ages, all sorts of use cases. And it needs to feel both natural and also kind of be correct. Whereas a startup has the luxury of not having to kind of meet that bar because the people who seek out and try a new startup product are kind of the natural early adopters, and they know that it's beta, they know that it's AI, they know that it's a test product. I think, I mean, not to give too much credit here, but I think what we've seen Google do well has been the new Google Labs experiments. And that's where a lot of the best, in my opinion at least, AI Google products have come out of, like NotebookLM and some of the video models like VO2, where essentially they have taken the approach where if you're an early technology tester, you kind of sign up and get on the waitlist and kind of beta test and use these products. And then, Ideally, they get them production-ready to launch to a slightly larger audience incrementally. But I mean, even as we've seen with NotebookLM, because it's Google, once it does make its way to the public, the maybe pace of innovation there is a lot slower than it would be at a net new company that doesn't have tens of thousands of people to continue to employ.

I mean, it's like 10 VPs for every engineer working on NotebookLM right now. So, and look, another example of this is Deep Research. Like deep research actually was originally a Gemini product and it's obviously a product that Google should be the best in the world at. And yet they never commercialized it in the right way for whatever reason. And now ChatGPT is known for their deep research capabilities. So it's just one missed opportunity after another with incumbents.

We'll circle back to that in a minute. Well, I wanted to get a little bit deeper on the stack and the sort of balance between allowing the AI to sort of handle things and kind of put your trust and confidence in its decision-making versus trying to maximise accuracy, which typically is going to mean more control measures and more kind of stilted, or let's say less natural interactions. The stack, for those that aren't familiar with this, basically has been the sort of audio in, transcribe that to text, feed that into a language model, feed that response back into another text-to-speech model, and then send the speech back to the user. Roughly speaking, you can complicate that if you want to, but that pipeline basically now does actually work fast enough in many cases to be viable. But then there's the fully all multimodal model, voice-to-voice, all through one set of weights. When you said that builders are mostly still building on these like more, you know, multi-part kind of pipeline style technology stacks, is that because they're getting better results from it? Or is it because like, that's the only thing that is like economical for their use case right now? Like what's driving that? And, and do you see that changing?

Yeah, I think the voice-to-voice models are definitely, and unsurprisingly, probably a little bit more expensive right now. But also I think they're just earlier, like we, I talk to a lot of voice agent founders who have probably tried all of the models available. And I think probably Gemini Flash is maybe the best of the bunch if you're going to kind of try to do full voice-to-voice. But in general, kind of like the interruptibility is not quite there yet on those models. But as we, I think, see and say pretty much every week, like the models are the worst that they're ever going to be right now. And so I'm sure similar to how last May, when we first released our voice report, like sub-1 second of latency was hard to imagine. I'm sure by this time next year, like everyone is going to be using a model stack that's much sleeker and is hard for us to even imagine right now.

I also think that there's, you know, reasoning models are a new primitive. And I think that they're maybe being underappreciated by many. Because they have the same interface as language models. But if you sort of think about it, it's like, well, there are aspects of interactions that really benefit from this probabilistic nature of the outputs of language models, all the friendship negotiation, et cetera, that you have in, in a business conversation even. And then there are things that really do not benefit from that where you want high degrees of accuracy, like a negotiation, like one price is definitively better than another for one of the parties and the counterparties. So in that case, you can sort of orchestrate reasoning models and language models to handle the right parts of the conversation to get the sort of desired output. So I think you have less of this, less of the issues around aspects of the conversation that demand accuracy and the models not being well suited to that. I think the other broad philosophical point, Mark, you know, Mark has mentioned this a lot, is, is the bar be as good as humans or is the bar be perfect? If the bar is to be perfect, like all these technologies are, are not there and maybe never will be. If the bar is to like outperform humans, I think we're actually there in many ways.

Hey, we'll continue our interview in a moment after a word from our sponsors. Yeah, it's, it's all happening very fast. How is the tool use? Because that's one other thing that I could sort of see being a challenge Although I could see it cutting either way, but depending on especially how idiosyncratic or esoteric your tool use context is, for example, if you're doing calls into some obscure freight management system, you might need to have potentially even the ability to fine tune that model to get those calls to work quite right. So I guess broadly speaking, how big of a deal is the Sort of backend interaction of tool use and like what's working or not working there today from what you've seen?

It's a good question. I mean, I think what we're broadly seeing is that you've got to build a lot more product than just the voice capability. I think the voice capability alone is insufficient. And, you know, there may be more sort of room to explore voice only in areas like agents where there's different types of conversations and You know, there's different outputs that you want and there's different considerations around sort of price point and, you know, what APIs you want to consume for what fidelity of output. But even then, there's an enormous amount of integrations and workflows that you probably have to deliver to build a traditional moat. So, you know, without commenting on a specific use case in something like freight, I just— our general observation is that like the capability gets you in the conversation, but isn't sufficient to get you to the other side.

Yeah, I would agree. I mean, especially when you think about a lot of these companies that voice agents are selling into are very much traditional traditional enterprises, like, they're not, you know, the Apples, the Googles of the world. And so, for them to build or launch on a more, you know, more horizontal way, or even to try to kind of build a voice agent themselves, like, in many ways, it's a miracle if they can do it once, let alone kind of keep it updated as the models get better, as, you know, there's new options that are gonna be a better experience for the customer, when the integration breaks with, like, the The kind of backend system of record, what do they do? And so, I think that is why we have seen so much maybe excitement on the customer side for more vertically focused platforms that, to your point, are maybe both fine-tuned for the types of conversations that these customers are having, but also have done the work to build out the long tail of integrations and also the long tail of honestly just conversation types of how do you manage kind of piecing together different tools for different types of tasks that you need to complete.

Yeah, I always think about Tyler Cowen saying context is that which is scarce. And that has never been more true than in the, I think he said that first before the AI moment, but the, or the AI era, I should say, but it's never been more true. And so much of what I see standing in the way between businesses that want to use AI and actual successful use is literally just assembling the context and sometimes getting the context out of their heads and onto some documented form that the AI can process. And so it's not too surprising that you would see base models, even as they are quite powerful and extremely versatile, certainly relative to anything that came before. It shouldn't be too surprising that they don't know the intricacies of not just the freight business in general, which they might even, but The way you handle your freight business, that stuff is, you know, that last mile, it trips a lot of people up. Do you have any sort of observations or like synthesis of what's going on there? I honestly still kind of struggle and I do like some amount of, not a lot, but like enough kind of hands-on consulting with businesses that I've seen this repeatedly where it just seems like people have a really hard time assembling the content. I mean, maybe this is what the verticalized startups are going to solve for us, but Do you have a sense for what's going on there? Because it seems like it should be easier, or we should be making faster progress in AI adoption than I feel like we're actually seeing.

Yeah, it's funny. I think we've seen definitely an explosion of kind of AI budgets within enterprises and within end customers. And in some cases, this was especially true 6 months ago. It's less true now, which I think is good, but it's like they're kind of looking for things to buy because to your point, they're not spending all their, all day around kind of what's the latest in AI. And so they're not exactly sure how to use it in their day-to-day. I think we even saw this with ChatGPT, where it launched, massive splash, fastest product ever to 100 million users. But then people weren't really sure how to use it every day. And the usage was flat for basically a year. And only now in the past year, as there's more models and more obvious things to do with it, has the usage kind of picked back up. So I think it's exactly to that point where on both the consumer side and the business side, like if people don't know what to do with the product, and if it's more than kind of a few steps to get up and running, like, there's going to be a decreasing funnel, unfortunately, of people who make it through and are actually becoming paying customers. And so that's part of the reason that I think that we're seeing companies that are so vertically focused see, like, the most success here.

Totally. And actually, I wouldn't underestimate the amount of growth these voice AI companies are seeing, like, especially on the agent But also on the Scribe side, like the, the, you know, the businesses either don't know how to use it or do know how to use it. And it's like growing explosively within these businesses because for the businesses that can, can kind of embrace it, it's such a conceptual substitute, you know, straightforward substitution for the humans that make the phone calls that there's no, you know, and of course they need to think then, okay, we're gonna hold it to some sort of a CSAT score and we're gonna measure outcomes on things like negotiations. So. Of course there's guardrails, there's, you know, integrations that need to be done, but, you know, in the cases where that's working, it's really, really working. And, you know, we're seeing some of the fastest growing B2B startups we've seen in 10 years.

Perfect tee up for a little lightning round on like what's really working across different corners of the economy. I was going to start with enterprise actually. So you mentioned Scribes, it seems like that's a pretty well established use case at this point. We're starting to see that bleed over into real-time coaching on calls and then obviously full-on agent substitution.

Is the real-time coaching working? What do we know about that?

And then is the substitution actually happening to the point where you think we'll see labour market statistic effects this year? Or do you think this will continue to be sort of isolated examples that are the exception rather than the norm for a while still?

Yeah, these are great questions. I would say that coaching is definitely working, and it is an interesting transition point in that there are some jobs, for example, a call center job, where if you're an AI product that is selling a coach into call center workers, there's a massive amount of demand for that right now. But what we might imagine that in, I don't know, 5 years, the— or probably less, the AI agent is gonna kind of replace a lot of those I think where the coaching will really continue to exist is these jobs that have kind of a heavy in-real-life component or heavy kind of personal component. We've seen quite a few AI real-time coaches for salespeople, for example, for HVAC technicians. Many of these jobs, like whether or not you get the $10,000 upsell, comes down to the nuance of what you say or the question that you ask. So, even if you're paying hundreds of dollars a month as an individual user for For an AI coach there, it's absolutely worth it. In terms of, like, the, I guess, the economic impact of the voice agents, I think, to be honest, like, there are some cases, a basic call center, for example, where an AI kind of taking the calls will free up the, like, human workers to do honestly much better and more rewarding jobs. Like, these are massively high turnover, 300% per year, like, Thankless jobs in many cases. And I think that, like, there's better things for people to be doing. And then in other cases, like recruiting is one example, we've seen quite a few voice agents that conduct initial screening calls for human recruiters. And that means that they can spend those 20 extra hours a week with the 5 candidates that they're really excited about and kind of catering to them and really convincing them to engage in the process and take the job. So in many cases, I think We see it as amplifying the humans in their current roles versus replacing them.

Yeah, I think all of that is exactly right. Potentially more humans sort of move up the stack to do higher-value work. I think the other thing is that you could take a look at what's happening and say, at the limit, we have 20% less jobs. Or you could say, at the limit, we all work 4 days a week. And we're paid to be optimists, but I believe that there's an opportunity for people to be more specialized and do more of the work that matters. less of the administrative overhead stuff that seems to consume most of our day-to-day.

Yeah, I'm with that. I guess what I'm trying to really zero in on though is like, you know, 'cause I think it's true in some ways what you're saying, but then there's other ways in which I think like, you know, retraining and reskilling and like everybody was gonna become a programmer and like that really hasn't happened. And I think when we look at sort of call centres specifically and the people that work there, we may free up resources and the company may grow and invest in other ways. But I think in many cases, the people that have the call centre jobs are not going to be moved into other jobs at that same company. And instead, the AI is going to just do that job and they'll just have a much lower headcount in the call centre operation. They may reinvest, there may be more R&D, there could be all sorts of great things.

Sure.

And by the way, I also think people should work less and, you know, sort of, you know, new social contract that embraces that is like, uh, high on my list of things people should be developing now. But kind of leaving aside like the second order effects of what happens, like, yeah. Are, do you think we are just like at a point technologically where we could see if enterprises wanted to do it, a like 90% headcount reduction in call centers?

I mean, I don't think so. We'll see. Right. So far we're not seeing it because as Olivia said, nobody's job at the call center is to just do initial phone screens. All right. People have recruiting jobs, which involve initial phone screens that are annoying and can be overwhelming, as well as, you know, deeper interviews, as well as salary negotiations, as well as ensuring that employees are successful once they onboard. So yes, the AI is going after the initial phone screen, but we haven't seen a reduction in headcount because all of the other work is so important. And, you know, frankly, in many cases, they don't yet trust the AI to do the work, or the AI can't do the work. You know, the AI can't take your employee to a baseball game a month after they've started and make sure they're having a fantastic experience. So, I totally understand the conceptual argument. It's just not something we've seen yet.

I think we'll also see kind of the success of AI open up probably like a new type of job that we haven't imagined before for humans to do. Like, one great example is one of the, I think, fastest-growing jobs right now is basically like contributing training data and doing online tasks and other things that that help the AI, which might be actually similarly paid, but a much better lifestyle than a call centre job in many ways. And so it'll be interesting to see what new types of opportunities open up for humans in the AI era.

Certainly the AIs are less abusive than the human callers. I think we can safely—

Yes, for sure.

That much.

Yeah.

Yes.

Yeah. I'm not, to be clear on my perspective on this, I'm not anti-displacement or trying to sort of still fear about that. I'm more— I think we probably are going to see it and should get ready for it. And ideally, it would be a good thing.

Yeah.

I often ask people outside of Silicon Valley bubble, and I live in Detroit, Michigan, if you didn't have to work the job you have to make the money that you need for the rest of your life, would you still work the job? And the overwhelming answer is no. And I consider myself very fortunate that I would probably continue to do what I'm doing Even if I weren't getting paid for it or didn't need to get paid for it. But I think it is really important for the sort of Silicon Valley set to kind of keep in mind that most jobs are not jobs that people are doing for the joy of the job. And if they could have their needs met in other ways, they would happily take that trade. So I'm not like a job preserver, but I'm just trying to figure out at what point is this wave of disruption actually going to hit? How much time do we have to sort of get ready for it? And it seems like the call centre thing, if I'm Understanding your answer correctly, it's like probably at least another year before we would see a sort of the— and I wouldn't necessarily— of course, these things are not binary either, but I put that 90% number out there just to sort of say, like, order of magnitude effect, even if it doesn't go to zero humans in the call centre. It sounds like you think that's like at least a 2026+ phenomenon.

Yeah, look, I think that it's cold to tell everybody, hey, just go learn programming. So that's not what I'm here saying at all. I just, I think it's very hard to understand what the labor impact of these technologies will be. And I think it's easy to sort of hypothesize about a world in which all the jobs just go away, but it's not what we're seeing yet. So even if the technology is 18 months away, I don't know that the labor market will change in the way that we're perhaps imagining as a result of the technology. I think we'll have to see. You know, I think a broad question though that you're speaking to is, What does it mean for our society when we have all this abundance? And is there kind of a lack of purpose? I have a big theory that kind of people need purpose, and if they don't have enough of it, they create it. And sometimes, you know, it gets pointed in bad directions. That's a lot of why I think Google ends up not working culturally. I mean, I think there's a lot of brilliant people there, but in a sense, it's sort of a, a low stakes environment from a purpose perspective because the business is almost too good. And I, I do get nervous about mirroring that in society. So I think an almost more interesting question to me is, Hey, in a world where we actually just do have all of this abundance, how do we sort of think about ensuring that people have purpose versus ensuring that people have, you know, jobs and income and all those other necessities?

Yeah, I'm a little more maybe even optimistic on that dimension, but it certainly, I filed that under a good problem to have. Okay. So this is supposed to be the lightning round. So we'll go through these next ones faster. So SMBs, you got the call answering. That seems pretty straightforward. Any like highlights or You know, for SMB owners out there, like, where should they go to potentially get, you know, the best of the AI call answers today?

Yeah, I would say, like, something we've been very excited about for SMBs is that there, even for SMBs, there's like a vertical solution depending on what you're doing. So if you're a restaurant operator, if you're, you know, a spa, if you're home services, we put also, we'll send our market map. Like, we, we put these all in their market map, but there is a solution catered to you, which is Fantastic. This actually gets back to what we were talking about before, which is like SMBs typically have one or two people doing nothing but answering phone calls, which is incredibly expensive for for a small business. And when we talk to the SMB customers, when they switch over to a voice agent, they are not laying off that human who is usually like a core part of their business. But that person is now able to spend their time doing things that are like much better for the customer experience or growing the business. further or, or other kind of extensions of the business, which are really powerful and exciting.

Okay. How about creators? This one is maybe a little less interactive, although maybe you're seeing interactive experiences that are sort of creator economy. Um, but one question I had, 'cause I might actually do an episode powered by AI voice in the not too distant future, who has the best voice design today? If I wanna create somebody that's gonna gimme that sort of film noir kind of Read, you know, where should I go for that sort of thing?

I mean, I think there's different answers to this question. Like one, you know, there's platforms like Eleven where you can clone your voice. Eleven also has a great tool where you can kind of like describe a voice, describe a sound, have it created. The other end of the creator part of AI voice to me is, is these like digital clones, which we're seeing more and more of, of platforms like Delphi, where you can essentially launch a version of yourself that your audience can interact with in your voice, or maybe via text or via other modalities, which is fascinating. I haven't seen AI replacing full podcast episodes for any podcasters yet. I think hypothetically we could get there where maybe you just like prompt the questions that you would ask. But we're probably still a couple, a couple years away from that.

It's worth though playing with HeyGen, Captions, a bunch of these other products to just, you know, you can fine tune a model of yourself, video and audio, and then giving it a script because I think there is a world in which You did do this entire podcast without ever owning a video camera or microphone.

Yeah.

Um, we did an episode with HeyGen actually, and for some reason Josh's audio wasn't great. And so we then redid the entire, his side of the conversation with his avatar from HeyGen. It's been ready like 6 months. So it was pretty good. I mean, it was not quite as good as, you know, the original would have been, but It, it was fitting that it happened on that episode.

Yeah.

How about for kids? I've been playing classic Nintendo games with my kids recently, and I put advanced voice mode on often playing Mario 64, the old like open world game, cuz I don't know where to go. Like, where's the star? What do I have to do? So I'll ask advanced voice mode like, all right, this is the level I'm playing. Where do I go? And my kids are now to the point where they're like, daddy, ask AI anytime that either I'm slow to do something or don't know what to do. So Daddy, ask AI. That's cool. I would love to have something interactive and educational for my kids, but I'm also like, yikes, I don't necessarily want to trust anyone to implement AI effectively for my kids. So any winners, any early winners in that space?

I think we've talked a lot about this as a team, and I think an area of exploration we're fascinated by is just all this stuff around behavioral, social, emotional, For kids. So I think that is an area where, you know, AI is very naturally suited to deliver value and there's just not a lot of technology there. So, you know, a great example is my son loves to play Minecraft, but all of the other people he meets online are like toxic teenagers. So, you know, why isn't there a companion that can sort of play Minecraft with him and model positive social behavior? You know, another example is just observing the classroom. If your child goes to one of those great schools where they've got 2 teachers in every classroom, one sort of doing all the academic components, one doing all the social-emotional components, you already benefit from this. But a lot of kids don't go to schools like that. So having a vision model, multimodal model that can observe the children interacting and give feedback to parents and teachers. So I think there's a ton. Of course, there's assignment generation and quiz generation and helping kids learn in whatever way is best suited to them. That stuff will happen and it'll be super important, but I think pairing it with all the kind of emotional opportunities is where we get most excited.

Yeah. I was going to say, I feel like we've seen companies like Synthesis and Elo and Super Teacher that are kind of like, what if every kid had a reading tutor, a math tutor that was sitting next to them all day and could kind of understand how they learn best and cater to them? And then on maybe the other end of the spectrum, we've seen things like the Curio toys, which is like, what if Maybe even more importantly than the tutor use cases for many kids, what if they had just a friend, a mentor, a coach that was with them every day and could both track their progress and help get them on the right track or even just be a completely sympathetic listening ear?

I mean, this is as a side note. Sorry, I know we're in lightning round. The companion stuff is so cool and it still feels— of course, there's so many amazing companies like Character and many others. bunch around the top 100, top 50, but it just feels like we're in the first inning maybe, or the warmup or something of exploring this space. 'Cause there's so many contextual opportunities to do it. And look, I think one, one aspect of it may be completely sympathetic. Another aspect of companions might be not that sympathetic, you know, one that really challenges you and pushes you and disagrees with you. So even something as simple as that, we always joke and call it East Coast mode. Like a companion that's like a little more terse doesn't exist. Why doesn't it? I don't know. And I think we're going to get to see those products in the next 2 years.

Yeah, that's really interesting. All right. In the interest of time, we'll skip over legal and medical, and maybe I can just ask for a word to Munjal from Hippocratic, because I would love to talk to him about what he's doing in the medical space. But how about just kind of You mentioned companionship. Maybe the last one in the lightning round is just kind of, it seems like the farthest out edge right now of this is going beyond companionship and into relationship and even not safe for work type of things. I don't know if that's stuff that you guys would touch in investment terms, but I trust that you're at least scouting that territory somewhat. What do you see going on on the far fringes of romance with AIs today?

I mean, it's a good question. I think the thing that is surprising, everybody maybe, or at least I had an assumption that most of the companion use cases would be, you know, like frisky young dudes. And it actually hasn't been that. A lot of it has been an audience that caters a lot more to women and probably feels more like interactive fiction than it does, you know, what you might consider pornography. So, you know, one, I think that there's a lot of mistaken assumptions that even I had about how the products would be used and who they're going to be used by. I also think that there's a lot of definitions of, of romance, you know, and, and I think that people are sort of perhaps critiquing these products saying that they're a substitute for traditional romance when in fact may make us so much more capable because you've either got somebody, you know, an AI that helps you train to be better at things like conversation and even flirting, you know, or an AI that can just be a vent for a lot of the frustration and, you know, emotional weight that people can sometimes bring to their in-person relationships. So those are some of the more surprising things. I don't know. What do you think, Olivia?

No, no, I agree. I think one of the— it's funny when we, whenever we pull the top 50, top 100 list of AI apps, like, and send it around to our team, like, every time without fail, people are like, oh my gosh, there's a ton of companion platforms on here. And a lot of them are maybe more NSFW or oriented, but I think it's been exactly what Anisha said in that it's actually much more of the AI boyfriend than the AI girlfriend use case, interestingly. And a lot more like interactive fan fiction maybe than anything else.

But that's a part of the human experience, right? Like sexuality is a part of the human experience and we, you know, we can't pretend it doesn't exist. And when we do, you end up as Apple who can't release a product for 5 years. So I think that we have to, you know, we just, we have to embrace that, that is going to be a part of these products and just, you know, find ways to, to get behind that. And of course, there's always going to be products at the fringes that we'll never invest in and perhaps most people will never use, but those are almost the least interesting products to talk about because it's always been that way.

Yeah, I've done 2 episodes actually with Eugenia from Replica, and the recent one was reviewing some research that folks at Stanford did that showed that not only did Replika reduce suicidal ideation in a substantial way for people that had that issue coming in, but also that more often than not, it helped people get out into the world. Question some of the data, some of it self-reported, whatever. But people indicated that they felt that using Replika was not holding them back, but in fact encouraging them to get out into the real world. So I thought it was quite interesting.

I think it's amazing.

Yeah, I'm a big fan. I think, as with all technology, but maybe even more so with AI, it feels to me like the shape and the specifics of the shape of exactly what we build is going to be really important. I have no doubt that you could make a predatory romantic AI that is addictive and exploitative in all sorts of ways. But I think we do see at least some existence proofs that you can make really positive, or at least predominantly majority positive versions of these things. And that brings me to a question on just kind of rules of the road. I know that it is early in this space. One rule that's been proposed is like, AI must disclose it's AI. That's a Yuval Noah Harari one that I like for its simplicity. I've also been thinking about the idea recently of a do not clone registry, which would be sort of the modern version of do not call. You could go and say, here's my likeness and my voice, don't clone me, other AI platforms. I'm wondering if you guys have any ideas for what either emerging best practices are or possible regulations that might keep all of this on the good side as much as possible for us.

Yeah, I mean, it's definitely early, but at least on my side, I've been surprised maybe by— it seems like more people these days are frustrated by especially the large model companies taking the approach of we're not going to let you do something versus people being frustrated that like I'm being deepfaked or I'm being cloned. Like that's not really happening to the average consumer right now. I think we've seen both the biggest startups and the biggest model companies be extremely careful about allowing you to even do anything related to a public figure, let alone like, you know, personal pictures or other things like that. And so I think I personally am very intrigued by the idea of of kind of the directory or the registry, especially because it opens up this opportunity for people of licensing or allowing their identity to be used for use cases that they are excited about. Like, we're seeing platforms like ElevenLabs, they have these kind of, you know, iconic voice collections of celebrities or people who will allow their voices to be used. But it's also been a massive boon to this industry of voiceover artists that maybe historically couldn't get like a job in Hollywood. And now there's all of these voiceover jobs on ElevenLabs. And we could see something similar in kind of the influencer or creator economy where if you're an influencer with, I don't know, 5,000 followers, you're gonna have a hard time to get a big brand to respond to you. But if the AI avatar version of yourself is even better and more powerful and more extensible, then, then maybe you actually can get some of those big deals. So I'm really interested to see how people can kind of extend themselves using the AI tools versus I at least have seen less maybe to be concerned about the everyday person who isn't a public figure getting deepfaked or anything like that.

I totally agree. I think every time there's a new technology, the kind of, you know, the talking heads try to get overly paternalistic. And I just don't think that's a generous enough view of the average consumer, how smart they are, how media literate they are. Of course, every technology has the potential for misuses. I'm not being glib about that. But I do think the paternalism is unwelcome and often unnecessary because people have learned for 30, 40 years that just because it's written in a book, it's not true. And just because it's on the internet, it's not true. And just because it's on social media, it's not true. There's no reason that this technology will be any different. And whatever we do here, I hope that we're sort of generous in our assumption that consumers are smart and savvy and will know how to use the products and technologies with like the appropriate level of caution.

Um, maybe just give me your like sort of medium or long-term vision for where this voice-enabled computing is going to go. Like, is it going to be Her? We're all walking around with the AI in our earpiece and we're untethered from our devices. Maybe we got glasses that pair with that. Like, what's the sort of You know, tech optimist view of like life in the, you know, in this voice-enabled computing future?

I think that we will see voice unlocked as a kind of modality feature on every product, in every interaction, in kind of every device. So, you know, AirPods, glasses, your computer. As we've dove into voice, especially from a consumer use case, you find that there's a lot of situations where maybe you don't actually want to be having a 2-way conversation, or you can't be having a 2-way conversation. You want it to be transcribing what you say, or vice versa. You can't talk, and you want it to be talking back to you. And so I think right now, we're in inning 1 of AI voice, where we have kind of a set of really compelling and exciting products. But 5 years from now, they're going to look incredibly limited based on what we have then, where you can interact with voice in any way at any time for whatever is like most useful and helpful to you.

You know, Steve Jobs famously said that a computer is a bicycle for your mind. And that meant that computer, you know, extended us intellectually in ways that were unimaginable. And that's what technology has done for us for 40 years. I think we're now going to have the emotional version of that sort of emotional bicycle where it extends us emotionally through products like companionship, but many, many more. I think voice is going to be the kind of primary catalyst and interface to that. So maybe a subject for our next conversation, but I think that's, that's really the way it's going to impact us. And it's been a bit underestimated.

Cool. I love it. Olivia Moore and Anisha Charya from a16z. Thank you both for being part of the Cognitive Revolution.

Thank you. It is both energizing and enlightening to hear why people listen and learn what they value about the show. So please. Don't hesitate to reach out via email at tcr@turpentine.co, or you can DM me on the social media platform of your choice.