Deborah Dahl: Chapter 1: How speech recognition’s past can inform our future

M-A-C-H-I-N-E-Y-E-S-T-I-X-T.

It's Machine Yearning from Assist. Welcome to another episode of the podcast that holds open the space for marketers, brands, and entrepreneurs to think, dream, and ask questions about the future of AI, the talking internet, and how we're reshaping our culture. A few episodes back, we brought you a series of conversations from the VOICE conference. With over 2,000 attendees at the largest gathering of the conversational technology world, VOICE was intense. You heard from voice and machine learning powerhouses like Cathy Pearl from Google and Patricia Scanlan, who has built a natural language dataset from over a million samples of children under 12. Now, we're excited to present another individual with profound natural language processing experience. Dr. Deborah Dahl. Deborah has been at the forefront of voice and speech, multimodal, and accessibility standards design on the web for over 30 years. Her view on the space and her sense of humor about it all is fantastic. These days, Deborah is the principal behind Conversational Technologies, a company that focuses on new disruptive applications of speech and language technologies. Let's dive in mid-conversation where we asked Deborah to step back and give us some overview on notable projects from across her storied career.

One big project that I worked on was a speech therapy system for people with aphasia. Aphasia is a language or speech problem caused usually by a stroke. It damages the parts of the brain that handle those jobs. Very common type of aphasia is when you look at something, you can't think of the word for it. So my clients had developed this this really great software that would show pictures to people and then they would have to identify what the picture was. But they didn't really have a good way of identifying it. So my job was to add speech recognition so that, you know, they could say bottle or water, cat, or whatever the picture was of, and then the system would tell them yes or no, you said the right thing. The problem is those users also had problems with their speech. Beach. You know, they might not say it right. They might have the right idea and they might say something close, but if they said "wata" instead of "water," it might not recognize them. So what we had to do, which is something kind of not you normally have to do, is we actually made a list of all kinds of speech errors that were common with the patients and said, "Okay, if you hear them say 'water' water, that's great. If you hear them say "wata," they still meant water. So it actually worked pretty well, but, you know, no funding. So it didn't get too far. But there's a lot of scope for these kind of rehabilitation medical applications of speech processing that are not really speech recognition and getting something done. They're for diagnosis or rehabilitation or assistive technology.

You worked on natural language projects for DARPA, which is the Defense Advanced Research Projects Agency. So most people know about their most famous outcome, which was—

The ARPANET.

The ARPANET, which predecessor to the web. But you were doing this in the 1980s.

Yeah, yeah.

I'm fascinated to hear about this. And like, what were the— there are a whole range of questions I want to get into with that. But sort of let's start with, you know, what were the projects that you were working on then? What were the problems that you were trying to solve in the '80s with DARPA, with natural language?

Well, that's really interesting. There's— I guess there was a jargon called dual-use defense projects, where there's a defense motivation and there's a scientific motivation. So you have an interesting scientific problem, which we started out— we were working on basic understanding of natural language, just completely independent of any particular application, just how do you get a machine to understand natural language. The actual projects we worked on had military implications. And they weren't military in any kind of scary way. They were text processing applications where we were trying to digest reports. So they're called CASREPs. They were reports on a ship of some machine not working. So they would write up a natural language description of that. All these reports would pile up, and then so what are you supposed to do with that? So we were trying to develop— as a test of our basic scientific goal, we were trying to develop ways of mass processing text information. And we had some other different kinds of things, which actually kind of led into my early work on air traffic control. It's not defense, but it's government, and we were trying to apply natural language understanding and speech recognition to making air traffic controllers' jobs easier, like reducing the amount of handwriting that they had to do. But something that was interesting was the early work that we did was completely text-based. And on the other side of the country, or at different universities, there was a lot of work done on speech. And kind of with the speech With speech recognition, the job stopped at the delivery of a text, of a transcript of whatever it was that was said. And with natural language processing, we started with the transcript and tried to figure out what it meant. And then around 1990, there was a DARPA project called— I think it was called Spoken Language Systems, where DARPA said, put these things together. And so you would actually have to start with speech, get a transcript, and then understand it. And that effort really cascaded. It was kind of the scientific underpinnings of the whole spoken language thing that we see going on even now.

So this is a question more about culture. I'm really interested to hear what people thought was possible then and what was required then.

Then?

And then I'd like to talk about how you've seen that evolve over time from the '80s to now the late 20-teens. So what did people think was essential at that point in time?

That's interesting because I think the things that they thought were problems turned out to be less problems. Can you give me an example? Well, real-time. In the '80s, real-time speech processing was you would be happy to get 2 or 3 times real time. So you'd say something and then a minute later it would get processed. That was a good result. And the other thing that we, in speech, that we thought was a problem was up until probably about the mid to late '90s, there was something called connected speech where you had to stop between each word.

I was an early Dragon Voice customer.

Yeah, yeah, exactly.

I know that experience very, very well.

Oh, it's horrible. And unless you had, if you had any alternative, you wouldn't. You wouldn't do it. But then people developed techniques for just understanding continuous speech. And that was a big problem then. The real time was a big problem. In natural language understanding, there are things that we overlooked, like domain adaptation, which means maybe I have a system that understands broken machines on ships. Turning that into something that you can order pizza with, that's not the problem. The big problem is getting an accurate parse. No matter what kind of hacks or engineering tricks you had to do to get your domain-specific application to work, that wasn't a real problem. The real issue is getting something to work no matter how big of a hack it was. I'm not sure if I answered your question. But I will tell you that I remember reading things from the early '90s where people were putting like 10-year-out projections and they were very, very sanguine about what we were going to be able to do. You could have a normal conversation with your machine just like you have with a person. I've heard that "just like talking to a person" many, many, many times over the last 30 years. I don't know, somehow it hasn't quite happened yet. They're even, like, the best Alexa experience is so far away from what a normal conversation is like. Maybe it's because we're all people and we all talk to each other all the time and it doesn't seem that hard. Even people with that technical background don't really internalize how hard it is to have a real normal conversation.

I think there are so many parts to this space in technological advancement that don't play out in the same way with others because it has to do with the intersection of human relatedness. And that it's a foundational element, you know, that when we look at people who struggle with it, who have developmental challenges, who might be on the autistic spectrum, who have different sort of relational, connective connection challenges. We look at them, and that's so personally threatening for so many people because we just grow as people who speak and have empathy and listen and feel and are related to others. And so I think the desire to have our technology do that as well for us is so powerful. And science fiction has done so much to fill in people's minds around all that because it's such a powerful urgent human desire. We want it so badly. We want to rush the future so desperately without realizing how much of our brain and areas of the brain we don't comprehend at all are working to make all of that happen. And so then to try to export that onto the machine, we still haven't fathomed how wide that gap is that we have to cross. You've been living at the front edge of this for 30 years.

Yeah, and I would say that, that one thing that's really hard for people people to separate is the emotional human-to-human connection from just capabilities. And like just able to understand a long sentence spoken quickly by someone with an accent, changing topics in a conversation quickly and going back to the old topic and remembering something that happened 2 days ago, all of which Alexa can't do. That's just the capabilities. And then there's, to me it's almost an orthogonal track, is making a connection. And I'll tell you something I just read about. It was a report by someone who makes some software that helps amp up the emotion in a machine conversation. And they got some survey results that said everybody wants a human being when you call up for technical support. You don't want a bot, or you don't want to be directed to a website. You don't want an automated system. And I would disagree with that. I think you want capability. I don't think you necessarily— well, actually, they said people are looking for an emotional connection with the call center agent. No, they're not. They're looking for someone to solve their problem. I'm sure you've run into call center agents that were trying to be friendly. And that just drives me crazy. Like, "Oh, how are you doing today?" I want to know why my order's not here. I don't care. I don't care if you care how I am. So I think we have a long way to go just in pure capabilities. And trying to compensate for lack of capability by friendliness is— I don't think that's a very good strategy.

When the limited range of emotion that we wind up having winds up being negative, somewhere on the spectrum of anger and frustration and being thwarted, you know, because I've said "help" an agent 16 times to the machine and it still can't figure out what I'm saying to it, it's like I'm having an emotional response. Response here. It's not the desire to emotional response.

And the machine says, "We're sorry, we're having trouble." No, you're not. I didn't get that.

I know you didn't get that.

The other thing about emotion is there's some ethical issues with trying to kind of manipulate people into making an emotional connection, especially children. I heard a story about a child that was sad when their family went on vacation because they thought Alexa would be lonely? It's certainly unethical to try to, like, try to pull something on users that might not quite understand what they're talking to.

Well, right. Well, I mean, that gets down to the area of persona.

Yeah, yeah.

You know, and how effective— I mean, it's kind of a good news, bad news. It's like, kudos to the people that designed the persona, but that you're talking about a person of a certain developmental stage that's not able to have the abstract relationship. With it. It's a literal, you know, that person of that age is having a literal relationship with that persona because it has the affect of a human being.

Right, right.

They're not recognizing that that's actually— they're not really knowing exactly what is it.

Just a bunch of wires. But you can see that also at the other end of the spectrum where something like Alexa is really a godsend for an older person that can't move around that well, maybe can't see that well, maybe they don't have anybody around to help them with things. But you don't want to trick them either. You don't want to trick them into thinking that it's really their friend.

Thanks for listening, and we hope you'll take a second to share this episode with other members of your team. Actually, make it easy on yourself, just subscribe. So you never miss an episode or a chapter. That way you can be the first to stay on top of this field and help shape the conversation at your company. Get in touch on Twitter @MachineYPodcast. DMs are open. We're super interested to hear who you think should appear on the podcast. Machine Learning is made by Paul Ciuffo and Michael Elsesser for Limina House. Have a great day.