Unaligned #9: Making "Her"

We're now able to feed in just an audio file and then you can ask it any question. You can say, what was the emotion that this person was feeling? What do you think their intent was? What type of music was it? What film would you imagine this music being able to be used in? Or you could ask it to write you a story about the audio content it just gave you.

Who are you?

Hi, I'm Trevor Back. I'm the chief Product Officer here at speechmatics.

Yeah. And tell us, let's set it up because most consumers know about Siri or Google Alexa, Google Assistant or Alexa. Right. And those are, I call them AI 1.0s and maybe they're 1.5s at this point, but they don't have. They're not, they're not smart. Like a large language model can make you, your system smart. Right. And now you're coming along. And how do you change that whole mix? Because there's a whole bunch of things starting to move. That's why I like talking with entrepreneurs like you who are building AIs.

Yes. Speechmatics has this incredible heritage in speech technology. It's been around for about 10 years. It spun out of the University of Cambridge, led by an academic called Tony Robinson, who's our founder and one of the leaders in the field. He's been working on speech recognition for 20, 30 years and so really knows the details of the things there. And basically there's this just enormous opportunity for every company, every person out there to really recognize the value in the audio data that they have. And companies like Google and Amazon and Microsoft have been offering speech recognition APIs, cloud, or through particular services like Siri or Alexa. But these systems are often quite brittle. They often don't understand every voice. And so our mission here at speechmatics is to really understand every voice, really focus in on that speech component, understand different accents, dialects, localizations, work on non English languages which are often underrepresented. And so we are really that focus play on really trying to make sure that speech recognition is keeping up to speed with all the other incredible breakthroughs in the AI space.

Yeah. So we should talk about. I had the same experience. I worked at Microsoft and Kai Fu lee showed me NLP like 20 years ago. Right. And it had a lot of errors, was very inflexible back then. It wasn't very useful now. And that was in a quiet conference room right. Where you had good microphone on the computer. Now, using something like chat GPT, you get whisper along with that, whispers the API that does voice and then you can use it at a rock concert and it understands you. Even when the microphone is like two feet from your mouth, even when there's a rock concert going down, it's, wow, that's really good. What do you bring to the table to compete with that or bring something new? Yeah.

So we've always focused on keeping the, what's called the word error rate, that's like the metric that matters in this domain, keeping that as low as possible and driving accuracy and really understanding it in every environment. And so there's an awful lot recently of new Whispers. A great example of these large multilingual end to end models that are focused on breadth over depth.

Yeah.

And so they really try and focus on being able to understand all the languages.

Yeah.

Unfortunately, despite the breakthroughs in AI recently, there's still no such thing as a free lunch. And so what these large end to end models end up doing is trading that breadth of languages for a la like lower accuracy in each of these. And so what Speechmatics has done instead is take a more sort of proprietary approach to really making sure that with high accuracy, all the languages and it means that we're able to use a much lower amount of data to get very high accuracy. And that then gives us this opportunity to ensure that less represented groups, people with speech impediments, challenges, accents, dialects. The UK has a lot of different accents and we ensure that those are represented. And by taking our particular approach, we're able to offer this to a broader range of users and consumers and get high accuracy. Still on that.

I have a special needs son who has a voice that's hard for a human being to understand. Right. He has a speech disability that makes it hard for him to speak and he's autistic. Right. And he doesn't speak very clearly for a lot of reasons. And he's frustrated by Siri and Alexa. They don't understand him at all. Right now Whisper is starting to understand him. It's getting closer to understanding him. Are you working on accessibility, understanding like that, like with people who have down syndrome or a heavy verbal problem, stuff like that? Are you working on getting that kind of data and training your AI to really understand people's voices that maybe are close to non verbal, we call it. Right.

So like we have three sort of guiding principles for us. Like we, we want to be the premium offering, so we want to bring the best accuracy. We want to be an innovative offering, so we want to make sure that we're keeping up to date with the latest tech and we want to make sure we're inclusive as well. I think that we, we took this approach which in machine learning terms is about being extremely sample efficient. You know, it's a lot harder to label audio data, it's a lot more.

Time consuming and my friend AJ has an auto tagger. Did you build your own auto tagger?

It's not like we have access to just all the text on the Internet. To train a large language model, we need to make sure we've got the right data, are well labeled and that's very time consuming and expensive. But because we go for this approach which requires less data, it means that when there is underrepresented groups, they consist of a larger proportion of that data set, which means that the algorithms tend to then work better on those underrepresented groups. We've got this fantastic example on our website where we have somebody that's deaf and so obviously they speak differently. And against all the competitors we do a lot better recognizing that voice and that disability. In terms of understanding the voice, our mission is to understand every voice. I don't know if we've got specific examples of like your child, but I know that our approach enables us to be able to help these types of individuals that aren't catered for by the massive models by Siri or Apple or Microsoft or Google.

So your approach to NLP understanding human voice is similar to a Tesla car or an autonomous car. The bleeding edge cars can drive down a street they've never been on before. So they learn very quickly about a new environment. So fast they can drive at 80 miles an hour through a new environment.

That's exactly, that's exactly right.

And you're trying to explain thing with voice, you're trying to learn about people's voices real fast. Tell us about what the breakthrough is here. Why are you able to learn faster? Why is your AI learning faster than other people?

Yeah, here at Speechmatics we've always had this like machine learning research group does focus on the foundational machine learning components that should be then useful. Yeah. And they're able to help translate breakthroughs from other domains into the speech domain for us.

Yeah.

And so over the last couple of years we've been picking up a lot on self supervised learning which is the ability to then learn from unlabeled data. So we have this huge model that focuses on just unlabeled audio data. Try and learn as much as possible from this unlabeled data set. Much the same as self driving cars. Try and just learn from all the image data they have. But then we have the second step where we then focus that model towards understanding particular labeled data. And this is our acoustic model. And that's where we provide this sort of this much smaller data set. But that is really clean, really well labeled, very precise, comes from like lots of different groups. And that's the way we are able then to tune this model towards understanding every voice. And so we're able to do that on over 50 languages now. But it means that even for languages where we only have maybe even a few hundred hours of labeled data, we're able to get extremely high accuracy, whereas the end to end approach would require a hundred x order of more labeled data to be able to do that.

Yeah.

And so our approach should be able to enable us to scale to other languages, other dialects, other regions, and potentially other disabilities and other things like that. So we're really able to focus on this understanding every voice, which is our guiding principle.

Now, if you ask Siri or any of these something like how many people are checked in at the Half Moon Bay Ritz Carlton on Foursquare, there it, Siri usually understands you just fine. In some situations it doesn't get the words right, but most of the time it does. But Siri is hard coded ish internally. I saw the back end of Siri when they launched it in my house, right. And you can see it's very inflexible architecture, very inflexible way of talking to APIs. Right. And we're starting to see that as a huge difference between like OpenAI and Siri. Because when you ask OpenAI the same question, it starts going out to Foursquare, right. And gathering data and trying to report a better answer than Siri does because it's not hooked up to the API. Foursquare, by the way, has an answer and has an API. It's just not hooked up. And so Siri gives you a stupid answer and goes to Bing and tries to get you an answer and it fails. And this happens all over the place on Siri. Right. It's very inflexible. You're still trying to do just the voice piece, right? You're not trying to go to Foursquare and get an answer and then give the answer. You're just trying to really understand the voice so that you give the next component a really accurate transcript of what's going on. Is that how I'm thinking about this?

Yeah, that's right.

Yeah.

That is the right way to say, I think There are enormous companies with huge budgets working on the sort of general purpose mind. Right. The general purpose AI, these large language models, and those are becoming increasingly available and accessible for everyone to use. But garbage in, garbage out. Right. So if you get the transcription wrong, if you understand the sort of what the words are being said, then you're going to get the wrong answer out.

Yeah.

But Alexa and Siri, they are built into that traditional brittle AI methodology that was more around like expert systems and more focused on very descriptive tasks that are like hard coded into the system. More large language models enables you to do is move from that brittle approach to a much more generalized fluid approach. I think you, you would argue that on a sort of chat interface using text and a keyboard, it's past the Turing test. A lot of people would argue. Right. But if you interact it with, interact with it with speech or voice, you can still tell that you're talking to an AI system.

Yeah.

And so the challenge really, I think for speech to be a core part of the future AGI is for that speech component to be as seamless and as human and as clear and concise and include things like being able to interrupt or being able to pause or being able to sigh.

Those really like, it really understands your conversation. Because you're going to be used by these new virtual beings, companies that are going to come out of the woodwork. Right. Personal AI, character AI. All these, there's 3,600 AI companies, a good portion of them are doing some sort of digital assistant, AI assistant. And they need you to do the voice to understand it. When you understand the voice, are you also understanding the context of the person? For instance, do you know that I'm in a grocery store when I'm talking to you? And do you add that context on underneath to the next component if you're sending a prompt over to ChatGPT to get an answer from your voicing, having a lot of context along with it helps make the answers better.

That's, that's the exactly the right question. Right. So I think there's an enormous amount of work and research and investment going into making these large language models larger, more general, able to understand more. But it really is important about what information is going into them.

Right.

And more and more companies are now looking at multimodal types of large language models or large. There's 26 of them.

They're raining out of the sky this week. Literally every few hours, another model ships. It's whoa, people are busy around the world shipping models.

I made a splash Right.

There's not just one, there's 26. Literally there's a list somebody's keeping. Oh, hugging face is keeping a list.

Exactly. But I think you really need to be able to interact with these in a very human way. And speech is one of the most human and most basic ways that we communicate, collaborate, interact with each other, et cetera. And we should be able to do the same with these AI models. And we're nowhere near the sort of sci fi future of how to seamlessly interact with these AI agents through our voice in the same way that we do through chat interfaces. Right. Like typing text into that side of things. And I think what we're focused on is how do we, how do we drive down latency in terms of the speed and how fast, how many milliseconds.

Take to give it to write a transcript and send it to the next component and the orchestra. Yes, you're going to be, be on.

Your phone soon is always focused in on its real time system. And so that's one of our core strengths.

But you're. You take a few hundred milliseconds to do work, right?

That's right. Yeah, that's right. So we do it in a few hundred milliseconds and there's sort of, there's a trade off between latency and accuracy. And so it's all about bringing down that trade off and making it faster, but maintaining the accuracy. And so that's what the machine learning model team here is really focused in on. How do you improve both of those metrics? But yeah, it only takes a few hundred milliseconds. But then there's also the latency of feeding it into the large language model, feeding it back out into a text to speech system and getting that back. And so it's, you need to own that full stack really to be able to pull down the seamless voice assistant rather than a seamless chat. As you say, like it's about the broader context, it's about the, the intent of the individual. It's about understanding and predicting what the individual is looking for for as well. And so we've been looking at how to use large language models and other AI methods to understand more than just speech to text, but also understand the emotion or the intent or music or other types of audio signals to really get a better context of what you're doing, such that then these large language models are fed more information through their prompt engineering so they can provide more useful answers back out. And so that's really, that's where I think we're heading in terms of speech first intelligence and clearly having that in real time is really important and interesting fact about speechmatics is that our sort of offline system, the model that's used there is exactly the same as our real time system. So any sort of breakthrough that we have goes directly into both of those systems. Yeah. So we've always been real time first for a good number of years now.

Are you only looking at the microphone or listening to the microphone or are you taking sensors from the GPS and other things, trying to figure out the context of the user beyond just their voice? Soon after we have glasses on with eye sensors. Right. They add context of what you're looking at, what you're talking about. Right. So that the system can get even more accurate at serving you in the future.

Yeah. So we really focus in on that speech component, on the audio component. There's just so much information like high temporality, high dimension, there's a background noise, there's other things going on, there's like different inflections you might put in the voice, understand the emotional content. There's a lot of research to be done there and I think the opportunity really for these AI agent ambitions is to that to be one part of this core stack. Right. So there's lots of other companies and people doing incredible work with understanding images, understanding text, understanding all the other modalities and I think the opportunity is for somebody to plug those all together. But here at speechmatics we think that speech is.

You're doing the one little model right there. You might have four models soon on your phone. Right. But you're doing the voice piece pretty much only. Right. But yeah, that's why I'm trying to figure, figure out how you're trying to get data about the user that might give better result later. Down in the, in the staff handed a better prompt. You're trying to build a prompt also with the text. Right. So that you can pass it on to the next component and pass along instructions and context about the, the sentence the user just said.

That's right. And I think we want to leverage the power of other large language models, but connect them up with the real time system. I think what we need to get to ultimately is this like always on streaming component, where the information from the audio is just constantly streamed into an LLM rather than put into a batch prompt and fed back equally. I think the LLM needs to constantly stream to a text to speech system and only then will you get this sort of seamless interaction and my. My. My sort of sci fi favorite version of this is Samantha from the movie Her.

Yeah.

The reason I go for that one is there's just so much seamless human connectivity. It's just you can tell that he forgets sometimes that she's an artificial intelligence. There's this amazing scene where she takes a sigh and he goes, why'd you do that? And she's. What do you mean? Why are you breathing? You don't need oxygen to survive. And she gets confronted by it and takes a step back and she says, I don't know. Maybe I just learned it from you. It's maybe just something helps me communicate. And he's, you don't need to breathe. Why do you do it? And the reason she does it is to communicate something to him. Communicate frustration or worry or anxiety and be more of a human.

Like when your virtual being is sitting on the couch next to you and you're talking with it, it has to act like a human being. It has to do human things like breathing. Right.

There are so many challenging when you're.

Talking to a human.

Yeah. There are so many challenging subtleties to like passing the Turing test in a voice.

Oh, we passed that long ago. Now we have to pass the new Turing test. Somebody. Stanford are coming up right now with a new Turing Test. Right. In fact, literally yesterday, there was a new evaluator shipped out of one of these schools. Right. For testing out all the LLMs out there and evaluating them for different purposes. How accurate is this? Little. Well, how accurate is this? There's a lot of engineering that's going to go into building these orchestras of AI soon that you're going to be the first stop. Right. You understand that microphone. So that's an important component. Are you. Do you see? Because you're not doing anything with the camera. You're not trying to be multimodal yet. You're just listening to the microphone. And you're specialized on that. And that's what you guys do really well. Somebody else will hook up your thing with a multimodal, open the camera up, and started adding even more context to the user and understanding the user better, understanding where they are, understanding. Because here's. I went to Spain a couple weeks ago, and I was sitting in, walking around Sidious Spain, right. With ChatGPT. And I took a picture of the church, and it told me all about the church. So it knows context. It knows things about everything around you. I shot food. It knew what kind of soup it was. It knew that it was an Iranian Soup. It knew how to convert that name to Iranian Persian. Right. It's crazy what's about to come. And you start thinking, if I was a product designer like at Siri, I'd be thinking about how to use something like you to make the voice really understood and hook that up with a multimodal model so that it would start doing magic things with the camera and understanding your context and then ingest the GPS and the other sensors also to further add on more context. So that when I finally go to ChatGPT or some other LLM on my phone for an answer, it really knows.

A lot that that's. Yeah, it's something that must be being invested in a number of different places. That combination. There's so many subtleties to the audio domain. Somebody somewhere needs to be focused on making that better. I think it works really well for English speakers in a room on their own over a really clear microphone. But imagine if when you're in Spain, you are chatting to. You sat at a restaurant and you're trying to order some food and a waiter doesn't speak your language and you don't speak theirs. Right. How do you seamlessly communicate and translate between each other? It's likely that these types of systems will be able to enable that, like real time translation as on top of understanding what you're saying and then being able to be like, infer the emotion that you're putting into it, not just the text.

Right, Here we go. You're starting to understand my emotion.

That's.

Do you understand that I'm yelling at the phone versus whispering to the phone versus Right. Stuff like that. Are you starting to put that as context? Like this guy's really angry, man with.

Large multimodal models are now enabling us to do more with our audio data. And so you used to have to train a specific model for understanding music, a specific model for understanding emotions, a different model for understanding intent and things like that. But actually now that we're able to. And we built a demo of this just last week and engineering team showed me this just last week. We're now able to feed in just an audio file and along with our text transcription and that audio file, we feed it directly into a large multimodal model. And then you can ask it any question. You can say, what was the emotion that this person was feeling? What do you think their intent was? What, what type of music was it? What film would you imagine this music being able to be used in? Or you could ask it to write you a story about the Audio content it just gave you. And this is where the sort of general capabilities that come from large language models can now be applied to directly to raw audio data. So I think we're in a really exciting inflection point for how do we get more information out from this very high bandwidth audio information that's not just what was said, but all the sort of surrounding information around it.

Yeah, let's talk about the business of this. How do you compete on cost? Right. Cause this is what developers are going to. They're going to be like, I need to build a voice app for my little whatever. And they're going to be looking at you versus a variety of others and trying to decide cost versus quality versus other things. How do you compete there?

Yeah, so we, we offer, we position ourselves as a premium platform. Right. So if you're looking for just any text out of audio, if you don't really care about the accuracy, if you're doing it for, I don't know, compliance reasons or regulation reasons, never going to use that transcript again. You're just doing it because you're being told to. There are other providers out there that might be able to do that for you. Yeah, but where sort of accuracy really matters in terms of recognizing all the keywords that were said, recognizing a range of different languages and different types of speakers, different accents and dialects, or where you want to then put that transcript into an LLM and you need it to be accurate because otherwise garbage in, garbage out, we're the most accurate. And so if you really care about accuracy, if that's important to you, if you're going to drive business value from that accuracy, then you'd be willing to pay for the most accurate model. And that's really where we drive like our interests. And that's where a lot of customers come to us. They use somebody else, they're like, does the job, but it's not great. There's lots of errors in it. That's fine for keeping it offline and never using it again. But if we really want to then derive value from passing them into a large language model and asking what were the biggest customer complaints from the last two months? You want to make sure that you're getting the right words into that system and in that sense you want the highest accuracy. And that's what speechmatics offers. And then the other side of it is real time. A lot of people Whisper is very good at doing offline transcription, but it's not yet, it's a large model, it's not yet Able to do really quickly at the moment.

12 seconds to get an answer back, sometimes 20, sometimes 4. It's usually a bit. Right.

It's, it's, yeah, it's like the classic if it takes too long to load a web page, the human brain wanders and goes off and thinks about something else.

And your assistant's not interactive. Come on, speed up. You're wa for the answer.

You really want that low latency, real time, fast response time. And that's again what we're able to offer. We're, we're the best accurate, low latency, real time speech system. And I think for the AI agents of the future, where we're going to want all of these Systems and these AIs to do 80% of our tasks for us, I think we're going to want to make sure that they're recognizing what we're asking, why we're asking it, how we're asking it. And we're going to want it to be really fast, super low latency and super high accuracy.

And this all runs locally, right? Or are you running in the cloud? Are you sending data to your cloud or is it all running on local phone or local desktop?

Yeah, so we actually offer like across a broad range of platforms. So we have a cloud offering for anybody that sort of wants all of that outsourced.

If you're on a bank website, you need to talk to the web, to the bank, right. Which you're really talking to an AI, right. You're. Somebody has to buy something like you or license something like you to do the whole system and make the bank serious.

Yeah, exact, exactly.

Customers are not going to the computer down because they got chat, GPT and Siri and Alexa and all these other things and that I, I doubt the big company's going to go with you. They've already decided what, which NLP system they're going to use or they're building their own. But there's a lot of business like the bank that needs to up with the Joneses. Right. Keep up with the Amazons.

That's right, that's right. And there's a lot of, there's a lot of customers out there. One of our other unique aspects really is we also offer like on premise deployment of our models as well. So there's a lot of companies out there that don't want to use the cloud or have data privacy or security reasons that they don't want it to go out outside. And so we offer our models on prem as well via the cloud and we also do on device. So, you know, there are companies out there that are using us on much smaller devices, not on huge GPU clusters or CPU clusters.

Yeah.

And one of the benefits there is because we've done our proprietary methodology that isn't this huge end to end model that's many billions of parameters, if not trillions, we're able to reduce the size down of these models so that they can run on device as well. And I think if I think about my iPhone and how I want to interact with an AI agent on that, quite often I will want it to stay locally on my iPhone. I don't always want it sent off to the cloud.

I have an AI listening to us right now from Rewind and it's just always listening. And it stores everything, everything on local. It doesn't do anything in the cloud. Right. It's all on my Macintosh. So there's a lot of privacy. It doesn't use a lot of power. Right. It's just sitting in the background doing work and it's really helpful. Right. It's really cool. And so a developer like that who wants to build some new thing for people, they can hook your component, your SDK right into their app pretty easily.

Yeah. So yeah, we've got an online portal where anybody can come and test us out and play with us. So it's really easy to just sign up and test us out and then we offer a range of SDKs for anybody that wants to move on to using us more seriously. I think for we're not yet at a state where people can download it and just put it onto their iPhone locally, but we do more bespoke deals on that side in terms of enabling the on device aspect. But I, I think that in the future that is going to be the direction of travel. Right. And yeah, so anybody that's looking to hack a quick system together and they really care about accuracy, it's very quick and easy to just grab our SDKs, plug into our cloud and just don't worry about how it's operating. Just send audio in and we'll give you very fast, rapid, real time transcription back.

I think in two years we're in a different world. In four, we're certainly in a different world. We're starting to get glasses from a variety of companies and we're talking to those things all day long. Right. Tell us, what do you see that future? Like two years. What's the big thing that we got to pay attention to when you're in the NLP industry?

The human brain Just really struggles with exponential patterns. Right. So we all think very linearly about progress. Exponential progress is really hard for humans to understand. And over the last decade AI has definitely felt like it's having exponential progress. And I think even me in the industry, I still struggle to think about exponential aspects of the progress we're going to see.

Models are raining down the sky. Did I tell you that? That's exponential progress right there. They weren't raining out the sky a year ago.

Right, right, exactly, exactly. It's the classic analogy of the grain of rice doubling on the chessboard. Right. When you get nearer to the end, it's like enormous. The sort of amount of progress you see in just couple of years, three, four years, we've seen like large language models go from a few million to a few trillion parameters. And it doesn't seem to be like that much slow in that speed of number of parameters. I think we do need to get way more sort of sample efficient on the data rather than just building larger and larger language models. But yeah, in two years time I do think that these co pilots are going to be just solving an enormous number of tasks for us, really doing a lot of our sort of administrative aspects.

Hey Optimus, pick up a screwdriver for me. Right, that's going to happen in the future. It might be four years, it might be six.

Right.

But it's coming. It might be eight in your house, it might be 20 in your house if you're really four. But it's coming.

It is guaranteed to become like the generalizable systems that we see today in AI is no longer. Can you just add this to my to do list? It's can go off, load up Amazon, buy me this item, get it delivered to me and have it solved for me. It's much more about doing those tasks for you than it is just about making a list for tasks that you should do in the future. And so in a couple of years time, these types of co pilots are going to be doing an enormous amount of our sort of at home consumer administrative tasks. But you're also going to see them in the enterprise solving a lot of additional problems, taking out a lot of mundane work that people do. I always laugh about the fact that these large language models are going to be taking a set of bullet points, making them into a long script, emailing them to somebody else, and then at that other side, somebody's going to be using a large language model to summarize that down into a few bullet points and then take that away. These Systems are going to be talking to each other a lot more than we as humans are talking to each other now. It's just going to be an exponential explosion, information passing around the world. And I think one of the core bits is we, we really need that to be less about us typing on a keyboard or using user interfaces and much more about like just having these conversational experiences because that's how it's going to be super simple for us to integrate.

That's why the glass line so important. You know, they're, they. This is an extraordinarily hard engineering problem, right. Apple is spend. Apple and Meta are both spending $40 billion building, building the glasses, building the custom silicon, building and building all the technology that's going to run these things as it's going to force everybody else to catch up. Because this weekend, if you, if your friend has a Vision Pro, you go over, oh, that's going to, that's going to change your idea of what a computer is. So now you want it now. Now if you're a business, you gotta keep up. You gotta spend some money building a system like that. And that's, that's right.

I think the large companies are in this crazy race to build bigger and more general like algorithms and models to be able to do more and more. But it also means that they're less focused then on the smaller components that then make up the overall like experience for a customer or a user. And so I think that's where like a company like speechmatics or other companies that are focused very much on the vision component or other companies that are focused very much on, I don't know, the user experience. There's a big opportunity there for these other smaller entities to really focus in on still really crucial parts of a future AGI stack and do them really well. There's this, there's a meme that like speech recognition is done, right. But anybody that's ever played with Siri or Alexa or Google Home or whatever knows that it's not even ChatGPT.

The voice feature is magic, right? But it takes 12 seconds to get an answer back. It's, there's work to be done and it gives you sometimes accurate, inaccurate answers, right? So there's lots of people trying to fix the accuracy of these AIs, right?

Exactly. And there's people that are being left behind. There's people like your child who, unless we're able to recognize their voice and understand them, they're not going to be able to leverage these types of systems in the Same way that a native English speaker and a clear background is able to benefit from.

That's a good question. Let's say my son still doesn't work. How much data do you need from somebody like my son with the same exact speech pattern to train your systems to really work for them?

Yeah. This is really the benefit of our methodology.

Could he read a card of 200 words and talk to the microphone and it all of a sudden it learns from his voice patterns that, oh, we need to do a little extra training. Right.

I think we would need probably a few hours.

Right.

Rather than like a few minutes and.

Probably very directed that he has to read a specific thing. Right? Yeah, yeah. You.

I did this with descript.

In fact, we're using descript to do our videos and I read to it for 40 minutes and taught it a lot about my voice. Right. Now you can type words and it speaks them like I spoke them. Right. It's really. It's the opposite of what you guys are doing it.

Right? That's right. That's right. Like, I think the breakthroughs on the text to speech component, where you're going from text to audio output, generative AI and that type of approach has really blossomed in the last couple of years and it's really made it possible to not sound like a synthesized computer.

You still do a little bit here and there. Right. But it's getting better. You can tell it's every year it's, whoa, that's getting better. Right. How many more years do we need before it gets really freaking amazing and you can't tell it's a human, not a human being anymore. Right.

I would guess less than two on that one. So back to your earlier question of what, like, what to see in two years. I expect in the next year or so it's going to be very difficult to tell if it was generated by an algorithm versus a human. Spoken like progress in that field is anonymous.

I talked with one entrepreneur who's building an AI for healthcare situ systems or et cetera. He built a, an AI for a nurses station at a hospital that a patient would call into. Right. And want some answers about their appointment tomorrow for a colonoscopy or something like that. I talked to it for a long time and it's hard to tell it's not human. It's really good. And it, it's really good because it's a constrained data set. And that's something that AJ who brought he. When I worked at Microsoft, a guy drove down and had a, a car that he could talk to. He was one of the first people who had put an NLP system into a car. And he told me it only can do a few things because I'm listening for a few things because that's how I get the accuracy rate up. This was 20 years ago. Is that still true today? That if you can you constrain the expectation of what the person's trying to communicate with you, the accuracy rating goes up still?

Yeah, that's definitely true. So we offer this thing called custom dictionary. You can add in words, you just type them in and it does an additional search across those text words to massively increases the accuracy. Imagine the use case of you're doing live captioning of a football game or a soccer game and you want to have the names of all the team players in there so that when those names are being spoken you recognize them as the team players rather than taking a guess. And that can have up to a thousand, ten thousand different keywords in it. And that really that constraining that sort of we expect these words to be said really helps drive up accuracy. It can work really well in you know, financial settings where there's very key terminology or clinical settings.

It's why the McDonald's ordering stations work so well. They know you're there to talk about a Big Mac or fries or a shake or a diet Coke. Right. You're not going to ask for, for a history essay of your McDonald's. Maybe it's because it's McDonald's actually work with a lot of people for ordering a Big Mac. Right. But it does that really well. Is your system if businesses are trying to build like if Chick Fil A is trying to build a new drive thru system, would they see you to build build the NLP for their ordering system in the drive through lanes?

They absolutely could. Maybe if you have some connections you can reach out and point of anoderation.

But somewhere I have to figure that out. I know I do because I had dinner with the head of innovation at Chick Fil? A one time. So I know he's in my life.

Hook me up sometime.

But you're driving another feature. You're powered by your company by Speech Matters. Right. Which shows. It shows there's a lot of business for people like you out there that has nothing to do with Siri or Alexa. Right. Even if Siri and Alexa to take all of that, there's still a lot of NLP needs around the world.

I think that large language models have had a huge impact on the AI industry as a whole, but I think they've had a particularly large impact on the speech in the past. The sort of number of use cases for why do you need a transcript of this audio file was much more limited. Whereas suddenly now with large language models able to utilize the large batches of unstructured data, which is essentially what a transcript is. Right. A huge list of wor words, very unstructured format, like you can quickly derive value from that very easily. And so the onset of like large language models has just increased the number of use cases for speech to text and speech recognition enormously. And so I think we're a very exciting inflection point for want, like people wanting more and more of these voice recognition systems. So what we're trying to.

Yeah, thank you. To finish this up, can you give me a little idea of the state of your business? How fast is it growing? How were you funded? How many people you got working for you? Tell us a little bit about the business behind this.

Yeah, so SpeechMax has been going for about 10 years now. It used to be a very small spin out from Cambridge, servicing just a few customers with quite old models. And as the sort of AI opportunity developed, it's continued to innovate and create new models. Now with 30 people servicing hundreds of customers.

How many of those are AI geniuses? Yeah, those kind of people.

Number of PhDs, there's a good like research bash. Like we're, we're located very close to Cambridge University and obviously our found comes as an academic from Cambridge University, so really well connected there. So we get a lot of great talent just coming out and joining us from that side. You know, we spend an awful lot of time and energy on research and innovation and how to leverage the best breakthroughs in AI on speech data. And so there's, yeah, there's quite a few ML experts really helping to drive innovation. We're continuing to look at the latest and greatest methodologies to drive down word error rate in audio data. Right. Yeah, we've got funding of around 60 million back in 2022, but we also have really great revenue that's helping us to reinvest and do innovation on that side. And so we, yeah, we realized a couple of years ago that there's enormous opportunity here given large language models for explosion and need for speech technology. And so we, we got some investment from some of the best investors to really accelerate our growth in that direction.

Very cool. Thanks so much for Ben. You gave us a deep dive in the world of nlp. The state of nlp. It's like it's changed quite a bit since I first saw it.

Yeah. So I, I started in the AI field back in 2012. I, I joined DeepMind as their first product manager when the company was about 15 people. I. The difference today of building AI systems is what it was like back in 2012 is just chalk and cheese. It's absolutely crazy how much easier it is to launch a product into the real world using an AI model than it was back then. It's. Yeah, it's a very exciting time.

Should we learn more about you, please.

Come to Speechmatics.com, check us out. Go test the demo on what we call the portal. Definitely try out the real time demo, the real time translation so you can convert your text into a different language. To convert your voice into a different language. We also just announced the bilingual model which use does English and Spanish. So if you can speak both English and Spanish, please go try that out, Test that out, give us some feedback.

I can't, but my best friend can, so we can try it out with him.

Excellent. Let me know how that goes.

Yeah, it's a fun world. Thank you so much. That's really crazy. It's a very important technology that most people don't think about very often, right. If they're talking to their phone or talking to their McDonald's at the drive through, they don't think about it, they're just ordering a Big Mac.

So I think that's what we want to do. We want to use speech recognition to push technology into the background, right. So that it's doing things for us without it interrupting our experience of the world. And we really want to head towards that place where AI is seamlessly integrated in our lives rather than taking over our, our lives. And I think voice technology can really push those interfaces into the background so that these systems can work without interrupting the lovely conversation that we just had.

Thank you so much. Appreciate it.