Tanay Kothari (Wispr Flow) at the Cerebral Valley Voice Summit

Whisperflow. I've actually at lunch spoken with a bunch of fans in the audience of Whisperflow. So a lot of people are excited to hear you speak.

So awesome.

In your own words, how would you describe Whisperflow as a product?

Today, the product's super simple. It's voice to text that just works everywhere. And at the heart of it, where we're going towards to build as a company, is Jarvis from Iron Man.

Jarvis from Iron Man.

Yep. Okay.

Starting with speech-to-text.

Starting with speech-to-text.

Okay. So when I think of Jarvis, thinking back to my time watching the Iron Man movies, there's a lot more going on than just transcribing what I'm saying, right? Yeah. He can do things. He can repair the suit. He can save Tony Stark when he's about to die. What are the steps between really awesome speech-to-text and Jarvis from Iron Man, and how are you getting us from here to there?

You know, when I think about Iron Man, it's less so about what it looks like in the movies. It's more so a system that just gets you.

Okay.

It's with you 24/7 and you trust it to do things on your behalf. And so how do you get to that from where we are in the world today? And it really breaks the company down in 3 phases. Okay. Phase 1 is you want to build a voice input that just works. It's reliable and people trust it blindly. It's important. If you don't build that, you can't build anything else on top of it. Phase 2 is Whisper doesn't just write for you, it's able to do things on your behalf. If you are a Whisper Flow user, you're gonna see a lot more happen in the product in the next 2 weeks. 2 weeks?

2 weeks?

2 weeks.

Oh man, come on, just like give us a little, give us a little early juice. I mean, it's only 14 days in advance. Like what are we gonna see?

You know, my PR team will not be happy if I said any more words. And then finally you get to Phase 3 where I can say, Max, you know this thing you love using on your phone or laptop? Imagine if you didn't even have to take your phone out of your pocket. That's when Wordsboro lives on everything from your smart glasses to smartwatches, smart rings, and beyond. And the thing we realized is you have to execute it in these 3 phases in this order, otherwise it doesn't work.

Okay, well, I have 2 questions about, I guess, the Jarvis vision specifically. First, I think, Right now, you guys have great speech-to-text. I'm a user. I'm a big fan. I want to wind it back to just focusing on that product. Because earlier on stage, we had, I think, Olivia Moore from Andreessen talking about how there's Aqua, and Willow, and Super Whisper. There's the dictation software on your iPhone, and Android, and everything. How did you guys— how have you broken out from that pack so far? And how do you feel like you'll continue to do so going forward? Because it does, it seems from a naive outsider's perspective like a market that is becoming commoditized or even from the very beginning was commoditized.

Yeah, yeah, yeah. For a lot of different models, going from a 98% to 99% accuracy feels similar, like if it's writing a poem for you or whatever. But in a couple of spaces, it matters a lot. Two of them are coding and voice, especially human input into machines. And so if Claude Code is 99.9% accurate and Codex is 99% accurate, you are going to Claude Code because it matters. And the same thing is what we realized with voice as well. And so voice-to-text on the surface feels like a really simple problem, but the thing is there are so many little nuances. That end up mattering. For example, one thing we noticed, again, the metric that we really shoot for is whenever you see a Whisper output, you don't have to make any edits whatsoever. Okay. People are very particular about commas and the way commas show up. Everybody has their unique style. We realize there's 15 different reasons why somebody would put a comma in a sentence. And so what Whisper does as you're using it in the first 5, 6 times is it learns your comma patterns. And so that really small detail is what adds to the delight. And that is one simple example of comma, and that's the level of obsession. And this is our ML team. This is not even the product team. This is what the ML engineers are obsessed about, is perfecting every little detail of what matters to people. Because if you nail that, then you build that deep sense of trust with the user, which gets you to the product love, the growth, and everything else that follows.

Yeah, I mean, I'm interested in that comma example specifically because, you know, I have seen Whisper seem to get better at sort of the flows of my particular speech. Is that something you guys are training your own model that learns from, or is it like fine-tuning based on my personal data? Like, is it a combination of those things? Do you have a separate text model that's sort of watching the text output that tends to come in for me on average and says, ah, you know, Max often puts a comma in here, even if that— I didn't detect that in the speech. Like, is it some orchestration of all of these things? Like, what is your sort of approach to even something as simple, but, you know, I agree, as important as, you know, punctuation and cadence and so on?

All of the above.

Okay, so all three things I named are part of the process.

Exactly. And the big thing that people don't know about Whisper is, uh, on the surface, super simple product for anybody who's used it. You press a button, you speak, and text shows up.

Yes.

What you don't know is it has one of the best ML teams working on the other side of it. My co-founder was one of the guys who invented diffusion models. Our most recent hire, our chief scientist, was leading all of voice and multimodal at Meta.

Okay.

For the Ray-Bans, Meta Reality Labs, and beyond. And he previously before that was the guy who started and built the entire Alexa team at Amazon. Okay. And so, okay. One of the highest-powered ML teams, I would say, outside of the foundation model labs.

Yeah. I mean, it does seem to work very well, and it does seem to get consistently better. And so I was interested in sort of the technical overlays of this kind of thing. But coming back to the Jarvis example from the beginning, right now it doesn't talk back, right?

Yep.

And a big part of the Jarvis experience is this butler, Jeeves-type personality with the British accent and everything that makes it— Jarvis into a character and not just an input mechanism. Like today Whisper doesn't talk back to me. It doesn't have a personality. Do you consider those to be essential elements of a voice agent going forward? And I guess the second question would be on the personality front, like how important is it to have custom personalities for each user, that kind of thing?

We will have a personality for Whisper.

Okay.

In terms of custom personalities, that is not, I would say, the main thing that matters. The main thing that matters is as a user, you need to feel that Whisper respects you, respects your time, and you need to feel like you respect Whisper. Because if you imagine Clippy, right? Clippy was the last thing that tried to talk back to you. You felt like, okay, Clippy's like—

I'm old enough to use Clippy, yeah.

Yeah, yeah, yeah. So if you're like, okay, Clippy's one. It's dumb. And like, two, you're annoying me and you're showing up in all the wrong places. You don't respect my time or my attention.

When I was writing a letter, it was great though.

I think you're older than me. I have less fond, flippy memories.

All right, all right, all right, sorry.

But you missed that respect piece. And so that I think is one of the most important things to build. Because if you feel that, then you give it a lot more permission to help you a lot more. You actively share a lot more data. And so at the end of the day, Whisper should really feel like a very smart chief of staff. And what happens when you have that is you slowly want them to know more about your life. You slowly want them to help you with more areas in your life. It can't happen day zero. It's more of like a progressive relationship building. And so when I'm thinking about the product, when I'm thinking about features or activation or whatnot, it all goes back to like, how would this experience be with another human being? Because that's what we want to emulate. Got it.

So yes, personality, maybe less custom elements to it for now. Yeah.

Yeah.

I'm interested, like, so I use Whisper all the time at work on my computer. I press the little button on the keyboard. But I will freely admit, I've been too lazy so far to install— I think you guys have a keyboard on iOS where I can use Whisper. I use the little Apple microphone button. And it screws up all the time. And I have to correct it and everything. But it's just there. And it's built into the keyboard. As a lazy consumer, I have just been using that on my phone. I'm wondering how you overcome that inertia, particularly I think on mobile, where people are less likely to install apps that take over different parts of their system. Where in a world where we have this iOS-Android duopoly, and both of them are pretty motivated, how do you overcome just the fact that that little microphone button's on the Apple keyboard and will be for everyone assumingly going forward?

So the question really, the way I think about it is there is a certain level of friction that exists and a certain level of value that is delivered. Now on desktop, the friction is low and the value is really high. And so you use it, you switch your behavior. On iOS, unfortunately, because of the Apple restrictions, the friction is much higher.

Much higher.

And so it is not that obvious of a choice. So again, two things to do here. One is, can we reduce the friction in certain ways? And so working on that. And two, can we increase the value significantly in other ways? Where again, this is all it is at the end of the day. Android actually is phenomenal. That is growing 5x faster than the iOS app. Because on Android, it's just a little floating bubble, nothing to do with the keyboard. You just press it, speak, works just like the desktop app.

Yeah, Android's a little more hackable, I feel like. Yeah, maybe that's not a good word, but something like that.

It is. Yeah.

I'm interested in the sort of work versus consumer use cases for Whisper, both today and in the future. I like using Whisper to dictate Slack messages, emails, that kind of thing. But I'll freely admit, in Silicon Valley, most of our team works in this giant open office. People are talking over each other. Some people find it annoying. It's a little bit of a library-type atmosphere in some cases, where it's like, hey, if you're too loud, go find a room and talk in private. What is your perspective on the workplace as a space in which people are going to use voice tools, given that inherent social fear of seeming annoying to your colleagues? And I think you and I talked about a US versus international split on this, which I'm also interested in. Interested in your perspective on.

So this is one of the first times where the interesting thing with Whisper is it makes you go from using your keyboard to using your voice, which is one, like, a big change to make on an individual habit change level, right? But also it's a big societal change for it to be okay to talk to your computer out loud and not look like a crazy person. This was very similar to the change that happened because of AirPods, when you started wearing them, walking around and talking out loud. When BlackBerry made it okay to have a little metal box in your hand that you look at all day long and write emails. Or Nike that made it okay to run outside, which was a very weird thing to do back in the day. And so when I think about this as both the individual habit formation problem and the societal problem that we're looking to solve. And so there's a couple of interesting things that started to happen. So one is, so one of our companies is a big, big Fortune 5.

Fortune 5 or Fortune 500?

Fortune 5. Yes. So one of the trillion-dollar companies.

One of the 5 largest revenue companies.

Yes. And so they have a 45% Whisper usage. Like 45% of the entire company uses Whisper.

As like DAU or ever?

MAU. MAU. Yeah.

What is that, like 50,000 people?

I'll say the number and then you guys— let me not get into that. Yeah, yeah, yeah.

Private conversation.

But about half the company uses that. The way we started it initially was just 500 people. And these people would just want to use it at home because they didn't want to speak out loud in their open office. But then one day we went there and on one floor we placed 5 mics. So when you went to the office, you were not the only crazy person. There was 5 other people like you. And before you know it, the entire floor was using Whisper and then half the company was using Whisper and all of All this happened in a period of 4 months. 'Cause we closed them in December and it's barely April.

So is your go-to-market just to distribute free mics to all the companies you're working with basically?

It's one of them.

One of them?

Why not?

Is that working? No, I'm honestly asking.

It's working phenomenally. And after that work, we sent it to 500 other companies. And at this point we're adding 2,000 new enterprise customers a month. Okay.

That sounds pretty high. So my last question, we're outta time, but like you guys are, it sounds like doing great on laptop, desktop, making good inroads on mobile. You mentioned a bunch of other devices, AirPods, smart rings, smartwatches, all this kind of stuff. What is the next device you are most excited about or you think that Whisper Flow has the biggest chance of being a huge success on?

I'm actually personally really bullish on glasses. Okay. Yeah.

From Meta or from just everyone in general, the whole space?

I think of us as we're the Switzerland here. We want to make voice experiences fantastic. And so I think Meta has done a fantastic job. So I'd love to work with them, love to work with a number of the other places out there as well. Because again, the funny thing that most people don't know is like we were not a software company when we started. We were a deep tech brain-computer interface company that was trying to build silent speech. And we did that for 3 years. We were trying to build hardware on how you use voice around other people. Oh, wow. And we built Whisper so that when you're in a world where you're just wearing a voice-first device, You don't have a screen in front of you, so you just need to trust everything that you're saying and the system's picking it up. You just need an extremely high bar for what that layer looks like. And we built Whisper for that. So Whisper inherently started off as being built for voice-first devices, and there's nothing better than the, the, the new glasses that are coming out as one of the first test grounds that is also like starting to become more and more mass market. Nice. All right.

Whisper on glasses. You heard it here first. Thanks so much.

Awesome. All right. This was great, man. Yeah.