VOICE Round-Up: Brooke Hawkins on the need for ethical design as we build the talking internet

It's Machine Learning from Assist. It's a podcast where we think and dream about the future of AI, the talking internet, and how we're reshaping our culture. This chapter is an excerpt from our interview with Brooke Hawkins at the 2018 VOICE Summit, the largest gathering of the conversational technology world. Brooke is one of the many folks we interviewed, people who are thriving at the front edge of brand, UX, research, and design. Brooke is a content designer at Nuance, and in this excerpt, she presents a compelling view on why an understanding of ethics, philosophy, and the liberal arts is going to be a competitive advantage in designing useful voice technology. This is just a taste, so make sure you grab the full episode and subscribe so you never miss a single Machine Yearning. Brooke Hawkins.

Creating frameworks for ethical conversational design.

Mm-hmm.

What does that mean to you?

It means to me kind of synthesizing expertise from other disciplines that I think we as technologists maybe have blind spots on. I think creating easy actionable steps that we as designers can take from disciplines like philosophy or ethics or different humanities really to embed and infuse into our technology work. Obviously we're all not going to be experts in ethical philosophy, but it would be wonderful if we could kind of create these frameworks and toolkits that boil down those concepts in ways that we can make actionable and then teach new designers that are entering the field, especially designers that come from a more computer science background to get comfortable and familiar with this stuff.

If you're positing that you can create frameworks for ethical conversation design, is the inverse that there are people out there doing unethical conversational design?

I— yes, I do think there are people doing unethical conversation design, but I don't think they mean to. I don't think any business or designers actively approaching their job trying to be malicious, but I think sometimes, definitely we're designing on behalf of large businesses and large institutions and we have goals and financial goals often that are driving why we make certain design decisions. But my hope as a designer is to create frameworks that challenge people to think outside of those tactical business goals and kind of approach things from a more humanistic perspective that can certainly take into account those business goals and achieve those, but kind of go a step beyond and make sure that we're being good human beings and providing patients and users and the people that are interacting with our systems resources to really improve their lives and perhaps challenge structural inequities that we either uphold or challenge through our digital tools.

You're doing work sort of training and developing new talent.

Mm-hmm.

Imparting some of these ideas to them. Let's talk a little bit about what you're doing with that and why you do it, why you love it.

Yeah, well, yeah, recently, most recently, one big education project that I've been working on is helping Career Foundry, an online UX course, create a specific VUI, or voice user interface design, course to train new talent that's interested in entering the field. Yeah, we tried our best aside from teaching the principles of voice user interface design to weave in some kind of ethical questions like, as a designer, what would your manifesto be around privacy for children? Or as a designer, what would your practice be around always providing things like outs or responses to things like help or exit or emergency or things like that? So kind of outside of just teaching designers the, way to design an effective system, kind of pushing them one step further to think about how to design a humane system. And I love it. I think this field is very new, and the work of training and supporting designers, especially diverse designers now, will be integral to the future of this field. I think if as designers we, in this voice space and conversation space, put out a lot of products that haven't been designed by diverse people, then the public won't adopt them, and they'll find them to be frivolous or silly or ineffective, and then we'll miss our moment. So I think it's a really important time to get a lot of diverse voices in the room and smart thinkers in the room to really bring these voice and conversational experiences to scale.

Let's talk a little bit more about how you teach people to have this active deliberation process and to spot, intercept, and sort of negate or off-ramp or just train out those limiting factors?

Well, I try to have, especially the clients I work with, think through the implications of the choices that we're letting users make. For example, I've worked with larger insurance companies to design virtual agents, and there are a lot of business rules and logic that I think just as a byproduct of the insurance agencies, how they work, if you have a lower education level or if you make less money or if for a variety of factors, you receive a poorer insurance rate. That's a perfect example of thinking through, well, we may not be able to challenge this, you know, structural inequity. This is kind of— this is how insurance works. But how in this conversation, as a real agent would, can we be a little radical and perhaps add a little bit more explanation about why a user is getting a poorer rate and perhaps some information about how they could receive a better insurance rate or why that structural inequity exists through conversation.

You see the credit companies attempting to do that. Mm-hmm. You know, it's a highly questionable business sector in some ways anyway, but when they're trying to explain, you know, factors that affect your credit score.

Mm-hmm.

Are we talking about things like that?

Exactly, yep. Mm-hmm. And I think just empowering users with information through a conversation, even when, yeah, we can't change the way credit scores work as conversation designers. That will take a lot of time. I want to eventually, but yeah, exactly. How through a conversation can we provide a user with actionable information to help them change the situation they're in, in a positive way?

With AI, you know, the errors are part of the intelligence. I mean, that's how everything is learning, right? But with these chatbots, these healthcare applications. I mean, this really does feel like we are, we are redefining the concept of high-stakes design beyond other areas that we've thought of as high-stakes design, which was airbags, physical safety things. This is more emotional, relational, intellectual high-stakes, but just as on the surface and urgent.

Cheryl Platz, a designer at Microsoft, just wrote a really cool article from a designer's perspective But these artificial intelligence tools like Alexa and Google Home and all these connected devices live in our homes. So necessarily they're part of our relational experiences. They're either amplifying structural inequities or not. And I think as designers, we need to think through how we could limit these structural inequities. But again, the example is in domestic abuse situations, which exist, these voice assistants are are now in these spaces where domestic abuse occurs. They're in the home, they're in these private moments that usually there's no other third party witnessing. And these tools are sometimes being used in kind of manipulative relationships to amplify abuse. This article specifically talked about partners that would randomly turn up volume on systems at unexpected times and kind of make the other partner live in fear or give the other partner this feeling that they're being constantly monitored by a third party. So As we've designed them today, these tools kind of amplify that problem. And now designers are having to take a look at that unintended consequence and think, how could we have designed, or how can we change the design or update the design of these products to empower the party that's being manipulated or hurt also now by our tools to kind of take back or radically change that situation for them? Those problems happen, those unintended consequences happen, and then we're in the position of, solving them retroactively, and it's clunky. We're kind of having to go back and add features or fix things that can kind of help, but I think what would be more radical, and as we're so new in this field, and what I think is really important is as new products are being developed, how from the beginning can we bring in diverse thought leaders and perspectives from the development phase of these new products so that that was never a potential unintended consequence? Perhaps if more women were at the table or more people with diverse domestic experiences were, were there when those products were being designed, they would have had the input or thought that, hey, this could have been used to negative ends. How can we prevent that from happening?

What we see right away is that introduces the idea of active monitoring.

Mm-hmm. Mm-hmm.

That, and you have the ethical rub of if we're doing active monitoring for good, meaning personal health and safety and harmony in the home versus active monitoring for monetization or commercialization or surveillance. Active monitoring becomes a real question in there.

Definitely, yeah. There, I think there have been a few examples so far of active monitoring being used for good. There was some article that came out, this family was having a domestic dispute and Alexa, through whatever processes, I don't know if it was confirmed how it occurred, but was able to call the police and bring in support to that situation to kind of squash it and help that family in that instance. There's been other instances of active monitoring where someone's private conversations were recorded and then sent as a voice memo to a friend unintentionally. So we're seeing slow examples of these kind of unintended consequences of active monitoring being a hero in some instances and in other instances being really creepy. And odd.

Terrific. Thank you.

This was fun. Thanks. Good.

Really good. I appreciate it.

Thanks for listening, and we hope you'll take a second and share this episode with other members of your team and make sure you check out the other chapters from this week's episode. Or what the heck? Just grab the whole thing. It's worth it. We promise. Next week, Kathy Pearl from Google. Bree Glaser from The Mars Agency, and two folks who are doing fascinating voice technology for kids, Patricia Scanlan and Avva Levin, all from The Voice Summit 2018. Get in touch on Twitter @assist. DMs are open. We're super interested to hear who you think should appear on the podcast. Machine Yearning is made by Paul Ciuffo and Michael Alsesser for Limina House. Have a great day.