Deborah Dahl: Chapter 2: From air traffic control to pricing crops, the potential of voice tech

M-A-C-H-I-N-E-Y-E-S-R-X-I-S-T. It's Machine Yearning from Assist. Welcome to another episode of the podcast that holds open the space for marketers, brands, and entrepreneurs to think, dream, and ask questions about the future of AI, the talking internet, and how we're reshaping our culture. A few episodes back, we brought you a series of conversations from the VOICE conference. With over 2,000 attendees at the largest gathering of the conversational technology world, VOICE was intense. You heard from voice and machine learning powerhouses like Cathy Pearl from Google and Patricia Scanlan, who has built a natural language dataset from over a million samples of children under 12. Now, we're excited to present another individual with profound natural language processing experience. Dr. Deborah Dahl. Deborah has been at the forefront of voice and speech, multimodal, and accessibility standards design on the web for over 30 years. Her view on the space, and her sense of humor about it all, is fantastic. These days, Deborah is the principal behind Conversational Technologies, a company that focuses on new disruptive applications of speech and language technologies. Let's dive in mid-conversation, where we asked Deborah to fill us in on some of her most recent projects, and she wowed us on the breadth of what's happening in speech-driven technology.

Deborah, tell us what you're working on today. What are your kind of current projects? What are the big things that you're thinking about that you're really excited about?

In general, I, I love to work on things that no one has ever done before. So usually they turn out to be really technically challenging, which is fine, it's great. So the biggest thing I'm working on right now is I'm working on a project for NASA The dream is that we'll be able to make drones able to understand air traffic control instructions. The drone is going to be, they call it pilot in a box, and it's going to have sufficient AI and natural language understanding and speech recognition that it can fly into an airport and be told what to do by the controller. My role is on the speech and natural language understanding side, and the speech is very, very, very challenging because there's an enormous amount of noise. There are speeches over the radio. And the air traffic controllers speak— I don't know if you've heard them before, but they are really, really fast. And so this is not like the routine speech that your Alexa hears in your living room. It's a completely different kind of cadence and style. So that's really challenging. And the natural language understanding part is we're just going to try to understand the words. What are the words? Well, on the positive side, it's a much more limited vocabulary. They don't just talk about anything. Sometimes they talk about kind of off-the-wall things like there's a bunch of geese on the runway or there's people on our airplane that have a short connection. But usually it's very routine, you know, "You get runway 36 left," you know, "Increase your altitude, go faster, look out for that guy over there." So that's positive on the natural language side. But then it is, it's a whole different language. So it's lots of really cool challenges. I'm not even getting into the part where the airplane has to actually be navigated by the controller's instruction. There's a whole bunch more challenges there.

Yes, I mean, so working with air traffic controllers takes the idea of working with lexicon up to a whole new level. And you just cannot import language from another —That's absolutely right.

That's absolutely right. They have all kinds of vocabulary that is not in normal vocabulary. And not only that, but if you think about flying all over, let's see, even just the United States, they're always talking about geographical locations. So there's the river and the bridge and the Highway 105. And there's all that vocabulary that's very specialized to a particular location, even though it's perfectly normal words. But like the Potomac River is only in one place. So there's thousands and thousands of words that you don't just get out of the normal dictionary.

So what's the process for building that library of language and then trying to start the teaching and training, you know, turning it over to the algorithms that it can start making that useful for the drones?

This is an interesting process because there isn't a lot of existing data. We're actually starting with a rule-based system, which kind of went out of fashion 10 years ago or so. But I actually think that a rule-based system is a good way to bootstrap a system that you don't have a lot of data for.

So explain a rule-based system.

That's— you write down the rules. You're not teaching it from annotated training data like a machine learning process. There might be a concept in this utterance, like runway or geese or some concept. So, but a human developer notices those kinds of things instead of where the deep learning or statistical processing more just gives a whole bunch of data to the algorithm and it comes up with a system. In our case, it's because the domain is very different from normal language. But you can also think it's going to be really important when we're— one of the big problems with challenges in natural language and speech recognition is languages that are not spoken by very many people. So there's— let's say you want a Yoruba speech recognition. Yoruba is actually spoken by millions of people, but where there's very little data for it. So I think we're going to find this kind of— first we do a rule-based system to kind of get ourselves going, and then once we can kind of automatically annotate data, create data that a machine learning program would need, then we can turn it over to machine learning.

Incredibly fascinating. What do you know about the sort of business implications and the business drivers behind bringing these drones into these airports and what they're really trying— like, what are sort of the steps 2 and 3 and 4 of introducing this? This activity?

Well, for the drones, well, there's a couple of things. There's a lot of airports without a controller, so actually, I guess in the long run, you'd want the drone to be able to talk to the other pilots, the sort of the ambient pilots. Currently, a drone that has to— not a drone in your backyard, but a drone that's in the national airspace, it needs a ground crew consisting of lots of really highly trained, expensive people. And I think the goal is to reduce that cost.

OK, that's just one tip of the iceberg right there. Oh, yeah. Yeah. Oh, yeah. Yeah. What's going on in Deborah's world?

Yeah. I'm doing a good job for NASA.

What else? What else is a big thing that's sort of really on your mind or occupying a lot of your attention right now?

I guess one thing that I'm worried about a little bit is— I don't know if you're familiar with the idea of the AI winner. That's happened a couple of times when AI got just so hyped up that it was going to solve all our problems, it was going to work miracles, it was going to understand everything we said, life was going to be so wonderful, and it didn't materialize. Then people got away from AI and said it doesn't work. Now we're coming to a new AI summer, where you have things like Alexa that are absolutely amazing. Their technical capabilities compared to what we could do even a couple of years ago are amazing. And people see that and they say, "Oh, Alexa is so smart and capable." And they don't realize that it really has a long, long way to go. We still have many speech challenges like accents and —like the air traffic control is people talking too fast. And the—

We were just talking to Patricia Scanlon, who's doing work exclusively focused on children under 12.

Exactly. That's a great example. And then there's all these thousands of languages that don't have enough speakers to really motivate that machine learning approach.

Or speakers who are at an economic position where there's not enough commercialization opportunity. OK, so—

So you may have an enor—

you have these enormous— I mean, Yoruba is a great example. Millions of people speaking it, but people from the outside looking in saying, "What's a commercialization?" Yeah, yeah.

And actually, there's a great need in places— I'm not sure about Africa so much, but definitely India— where there's a literacy problem. And so what could be better than speech if you can't read and write? We have handset ubiquity.

We have near handset ubiquity. We're pushing as aggressively as possible toward that. So you have a microphone and a playback in many, many people's hands all around the world. And so the opportunity to do health— Yeah. Literacy. Yeah. Politics, civics, you know, all these sort of fabrics of societies that could be mediated through the handset. Yeah. If NLP and AI were really mobilized, I think especially, you know, sort of economically underdeveloped areas are particularly ripe for reaping the benefits?

I actually gave a couple of voice classes in Africa a few years ago, and I learned a lot about the kinds of things that are important to— maybe not Africa per se, but like rural communities. So they— sometimes the transportation is terrible. They have— they're farmers, and they have a crop to bring to market, and they really need to know what the prices they're going to get for their product. And, you know, maybe they can call somebody, but maybe there isn't anybody they can call. But wouldn't it be great to have just a really simple service where you could find out what the price of rice is in these 3 different towns? Otherwise, they have to invest a whole day in going to the town and maybe sell their crop at a, you know, not the price that they'd like to get. And I think agriculture is a great area that is maybe overlooked a little bit because all the technologists live on— they don't live on farms. But there's a lot of opportunity in areas like that. There's, I guess, a combination of lack of imagination and lack of financial motivation. You know, people, you know, they want to make money from their product. Yeah, yeah, absolutely.

Deborah, I really appreciate your time.

Thank you. Well, thank you.

This is really fun. Oh, good. Thanks for listening, and we hope you'll take a second to share this episode with other members of your team. Actually, make it easy on yourself. Just subscribe so you never miss an episode or a chapter. That way you can be the first to stay on top of this field and help shape the conversation at your company. Get in touch on Twitter @MachineYPodcast. DMs are open. We're super interested to hear who you think should appear on the podcast. Machine Learning is made by Paul Ciuffo and Michael Elsesser for Limina House. Have a great day.