Robin Sloan: Chapter 2: How To Not Make Autocomplete Dystopian

M-A-C-H-I-N-E-Y-E-S-I-S-T.

It's Machine Yearning from Assist. It's a podcast where we think and dream about the future of AI, the talking internet, and how we're reshaping our culture. This chapter is an excerpt from our longer conversation with Robin Sloan. Robin is a sci-fi writer, social observer, and dedicated AI enthusiast. He's been bootstrapping his way into some fascinating co-writing experiments with the machine. Since we've been talking about writing as a craft, the challenge to improve your craft was defined in a few ways. Now, as Robin Sloan is describing, machine learning, the tools and ways you have to stretch as a writer and collaborator, have changed the game, especially as we learn to collaborate with the machine. This is the essence of this podcast. Machine Yearning is all about the ways we navigate a new world where identity and technology and commerce and security have to live together in ways we can't even fully imagine yet. Enjoy.

I want to know more of what the tool is.

Sure.

Because I don't understand. I mean, I guess to take the listener there and you there.

Yeah, yeah.

I want to know what is this tool?

Yeah, yeah. So basically it's, and this is where, you know, You know, it's gonna kind of expose the limits of my own understanding because I'm not one of these machine learning researchers, but I have used the tools a lot. It's almost always, not 100% all the time, but almost always something called a recurrent neural network. And it's actually at its heart, it's kind of beautifully simple. It's a neural network that learns about patterns, about sequences, and it could do that for musical notes. It could do it for numbers in some weird cryptic series that you wanted to understand, like maybe you're trying to model a weather pattern or something like that. It can definitely do it for text, right? Because that's just text, is always some kind of sequence, a sequence of words or sequence of characters. And really all it learns, I mean, you could, you can imagine if we were just doing something simple on pencil and paper trying to come up with a model of like English, we would determine really quickly that given the letter T followed by the letter H, it's very likely that the next letter is going to be E or A, very unlikely, like vanishingly unlikely, that the next letter is going to be Z or Q, right? So you imagine that as a little kind of like module of statistical understanding.

But the engine doesn't understand English.

That, I mean, that's actually the beautiful thing. It doesn't understand English. On moment one of like training this recurrent neural network, it's not like you've coded it saying, okay, get ready, you're going to break this up into words and here's what a sentence is.

It doesn't even know what a word is.

Oh, no. No idea. It is just like opaque data.

Does it see the T as a line that's horizontal and a line that's vertical? Yeah. Or does it see the T as a letter?

It sees the T as a letter. It sees the T actually— I mean, usually the way it sees it is as a number, as, or as one in an array of, you know, 128 potential values or 256. It's really—

Interesting.

And that's, that's the astonishing thing. Starting from that kind of really, really cold start of just having no preconceptions about language or the world and then training itself on— again, and this is not like one short story or even one novel. I mean, to generate kind of to bootstrap this deep understanding of the structure of English and the way that people write. The experiments I've done that have, to me, been the most successful in terms of generating the text that's been the most interesting and like the most interesting to read have been all with a big body of text that I collected from public domain scans of old science fiction magazines. And so it's hundreds and hundreds of megabytes of old science fiction stories from like the '60s and the '70s, all kind of glommed together. Lots of different writers, different eras, but all with that, you know, that set of subjects and kind of that tone, that science fiction tone.

So apply it, and then you use that how? Like, you're sitting there at your computer, you're typing your fingers on words and let— or you're typing your fingers on letters.

Yeah.

And then you have this robot that sits in your head, like, and you hit, "Robot, help me." Like, what happens?

Well, so that's— and for me, that was the second step. It— initially, because of course we're just using these tools that kind of researchers have developed, it's all happening on the command line, and I didn't actually see how I could write with that. It was kind of a curiosity. It'd be like, "Whoa, look! The computer just generated, like—" 2 pages of really weird, interesting text. Okay, I'm not really sure like exactly how to use that. And so I did— it was— I was riding a BART train actually coming into San Francisco. This is maybe a year and a half ago now, almost 2 years ago, riding a BART train coming into San Francisco thinking about this stuff because I just kind of started tinkering with it. And I had the notion that what I wanted was really autocomplete. I wanted a super creative autocomplete. I wanted to be able to start typing something myself and hit Tab or some key and have this system, this statistical model of, you know, in this case science fiction, complete the thought for me. And so that's what I ended up building. Interesting. Yeah, and so, you know, you could type, uh, the ship landed on the planet's surface. Captain Astro stepped down and saw— and then you're like, well, what did she see? I don't know. 'Let's see what— let's see what the computer thinks.' Hit Tab and it'll fill it in. And it— I mean, and it could say anything. What's good though, what's cool is that it's not garble.

So how are you thinking about this phenomenon? How should I think about this? What, what is like— where is this going?

Well, you know, I think there's kind of two branches for these kind of technologies. And when I say these kind of technologies, I guess I mean programs that have an increasingly sophisticated understanding of how we write and how text comes together and can kind of fill in the blanks in interesting ways, right? And I think, I mean, obviously autocomplete exists and is widely used everywhere. So that's— fucking ruins my text every day. That's the— I'm totally— I text sci-fi a lot, like a lot, at least a couple times a day. And for some reason autocomplete has not figured out that I want to say sci-fi. It always says sci-do. Which is not a thing. I don't understand it. Said you. So that exists, obviously, and it proves that, like, these are gonna— these tools are gonna be an important part of our life— lives. I think there's two branches now going forward. One is, to me, a little darker and more dystopian. The other one is more interesting. The darker, more dystopian one is a future version of Gmail where it pretty quickly understands what kind of email you're writing and basically offers to write the majority of it for you, or to like make a pretty good draft of this email for you to then go back and edit and then send.

That's dystopian to you. Yeah, it is. Oh yeah, it is.

It really is. Just only because, only because I really— maybe this is just— No, I get it. You know, but it's because I think it's because of just who I am and how I make my living now. The idea of words that are essentially signed by a person, words that pretend to be from another human, but they're not really. They're essentially like a form letter generated by a computer. I just think that's gross. I just think it's kind of an insult to inflict that on another human being.

When does your dystopian lapse and you're like, fuck it, it's just, that's just how it is? Or does it?

Well, you know, it— that's a good question. I don't know. I have tended to focus on the way, in a way, almost heading off the dystopia. And this is what I've done, like, in the tools that I've been building I've been trying to sort of produce and share implementations of these techniques that are designed not to like make the world more efficient and, you know, make communication faster and easier, because I frankly don't care about any of those things, but to make stuff weird. To make stuff weird. Yeah, to like, to kind of, to weird it up, to add a little perturbation of strangeness and like, whoa, what did I just read? Or like, what is going on over there? Give me an example. I mean, the example is in this— Give me some weird shit. I want some weird shit. I should have brought I should have brought a short story to read. I mean, the answer, the weird shit is in sort of being in conversation with these machines. And in particular, machines— this is the thing I think a lot of machine learning people in both research and industry overlook. They focus on the engine, assuming that the thing you train it on is going to be something big, broad, generic, useful. I mean, imagine all the emails stored in Gmail, or imagine a just tons and tons of, like, news articles from Reuters or the New York Times. This is the kind of stuff that people use. As we discussed, the things that I've tended to use have been old sci-fi stories. I've done experiments where I've taken two different bodies of text, really different, like one kind of horror fiction and another the text, the transcripts of Supreme Court proceedings, and mash those together. And told the system like, all right, learn how to generate text that fits both of those at once.

Yeah, and, and the goal— like Republican and Democrat? Absolutely. I mean, I mean, like press releases.

The truth is there's a ton, there's a ton, there's a ton of interesting work to be done in thinking about what we train these systems on. And I think that's gonna have to end up being the, the kind of domain or the work of a lot of writers and artists and weird people, because I just don't think Google or like Stanford grad students are ever gonna do it. They're just kind of focused on different things.

The efficiency game is what it's used for today. Yeah, that's right. You're focusing on something that's, I think, way bigger, but everyone sees it as like an artistic kind of thing. But it feels like everyone in the human side of things— we're coming out of 100 years industrialization. We're like, okay, those are probably gonna be lost jobs, maybe. And if so, then the only thing that matters is creativity and creative thinking. If the machine becomes creative, what happens?

You know, it's interesting. Just to— I mean, that's a big question, a big thing. It's a big thing to think about. Even to just narrow it down and think about text, you know, what if I wasn't working on tools to kind of augment my own writing or the writing of a person? What if I really was making a machine that could write a short story? And that's not implausible. I mean, there's the pieces of the puzzle are sort of there, and you could imagine with a few more steps forward, just a few kind of increments, I could imagine a system where I say not just finish this sentence, but okay computer, write me a short story, and it writes one and it's actually pretty good.

Have you heard of the infinite monkey theorem? Yes, yes, I just heard about this two weeks ago. Yeah, that's Robert. Yeah. Yeah. Yeah. Yeah. He's like, here's the deal, right? If you have infinite monkeys and you give them a keyboard or typewriter and you let them just type One of them will write a Shakespeare novel.

Yeah, yeah, yeah. I mean, I'm like, what? And I read about it and I was like, what is going on? I mean, what it actually, in a way, it loops back around because what that leaves out is that you need to identify it somehow. Somebody would have to be sitting there reading the output. This sounds like a great job, by the way. It's like somebody's gonna get stuck with this job, reading the garbled output of the infinite monkeys until they're like, "Yes, here it is. Here's Shakespeare." Um, at which point you're like, maybe we should have just had that person write the Shakespeare in the first place or whatever. And I think, but I think that's important. Um, it turns out the people who are to me smartest about this stuff, and it's not just the sort of the text-based, you know, machine learning experiments. There's a lot of people doing work with images, people doing work with sound and music and all the people who seem to me smartest and most grounded about these tools, they insist that there is this super important step at the end, which is a kind of filtering or editing or curation. You essentially ask these systems to produce a bunch of material. It's really weird, interesting material, but like, that's not the end of the process. You as an editor, as an artist, as a thinker, as an editor, have to sit there and say, "Mm, this, not that." will use this, that we'll throw in the trash can. But it's, I guess I would say that's one of my sci-fi predictions I feel most confident about.

What, can you make that prediction very clear?

Yeah, in some number of years, I don't know if it's 5, I don't know if it's 10, most text editors that people use in email, in their offices, are going to have totally augmented writing capabilities. And you're just tweaking it. Yeah, exactly, you're the editor. You're kind of building a document block by block. And then, and then yeah, going back and making some changes or saying, oh, that doesn't sound like me. Or, but then again, maybe you just select a whole paragraph and press the key that means like make more friendly and it adds more exclamation marks and maybe a few emojis. Yeah, yeah, yeah. I think that's, I think that's, it's coming for sure.

And now my email marketing software, my blog software, my chatbot software for messaging, my Alexa, software, whatever is powering our language, it's suggesting what it knows I already should do. Yeah. Yeah, probably tested it too. And I'm just like, I'm more of like the legal department approving it.

Yeah, and I mean, in a weird way, but then, and it raises the interesting questions because of course, you know, humans, we always, we like get these tools and then the tool, the tools change us and they change our behavior. It raised the question like, then will there be this little thing where you'll write this email to your colleague using this robotic system but then you'll put some little thing at the bottom that's the little wink of life. You know, there'll be some way of saying, "Hey dude," or a little emoticon, or some like secret code word that becomes evidence. Like, it's because it's something that the robot could never write. I don't know what that would be, but it does— I mean, if it's all a perfect masquerade, then some people are gonna find a way to poke through or a way to undermine that system. It's gonna get weird. I think it's— it already seems very weird to me in the era of like Slack and everything else to work in a very tech kind of saturated office.

I think it's just gonna get weirder and weirder. Fascinating. I was at this talk one night. It was people from IBM Watson, and they were showing how they can see cancer in the photos. They can— the machine can learn if it's cancerous. They claim it was like 99% effective. The human error rate for detecting cancer was like 80%, yet the human was required to make the decision at the end because we still trusted the human who had a worse error rate. Yeah, and then they got this huge argument. One, I think the Watson guy was full of shit, totally full of shit. But two, that's why I think you needed the human. Yeah, the other guy in the room was like a real scientist, and he was like, if you believed in it, And the percentages are right, you're gonna kill 19% of people because you don't believe in the machine. And I was like, fuck. I like blew my mind. I was like, I don't know what I would do here. There's—

I— boy, we are gonna be faced with more and more of those challenges. It is— whose fault is it? It's deeply uncomfortable. Yeah. Well, I mean, I actually— I think one of the reasons— this is pretty gnarly— but actually one of the reasons that humans stay in the loop in a lot of these cases is that it's not clear. If there's not a human, it's not clear whose fault it is, or worse, for a big company like IBM or Boeing or whoever, Tesla, the people who make this autonomous software, if there's not a human in the loop, it might mean that the fault is theirs. And like, they don't want that.

Yeah. Who's the insurance go out to?

Exactly. Exactly.

Do you think that's real? Like, that's a huge—

I do think it's real. That's going to happen. Yeah. There's a, there's a phrase kind of bouncing around a little bit in kind of AI policy circles. Um, the idea of a human operator or a human, a little, little bit of human judgment as a, kind of a moral crumple zone, you know, the person's the one who can absorb the shock of something going wrong. And it's not that we would never figure that out. There's going to be new policies and new precedents set around liability in terms of AI decisions. But it's super unsettled right now. And I actually think it is just easier for a lot of, for a lot of these companies and institutions to be like, "Yeah, it's, it's Robin. He was the one driving," or "He was the one who like looked at that screen and pressed a button that said," Okay, I approve.

The limiter to advancements in the technology is because I need someone to blame. Yeah.

Whoa. I mean, that's gonna change. That's gonna change, but it's such early days now. You know what? Nobody knows anything.

My question to you would be, take— that's like a very dark version.

Yeah, yeah, right.

But take it in a world that's, you know, very politically correct today. We're politically correct, right? It's really hard to not be politically correct, although some people aren't, and then the internet goes crazy. Do we care if robots are politically correct?

Do we care if the sentence it produces sounds sexist, racist, or terrible? Yeah, I think we do. I think we do deeply. There's two pieces to that. One is, of course, they only learn what they're taught, and we've seen this time and time again. Yeah. Training sets have, of course, you know, of course they have biases built in. They're human artifacts. They don't just kind of come from the cosmos. They are assembled, put together, and fed into the computers by people. So, so to the degree that what comes out the other end of these systems is biased or weird or like privileges one group of people or one set of ideas over another, I mean, that's actually telling— just telling you something about our world. It's not telling you too much about the machine itself. But the other thing I'd add is that You know, I think it's like using a machine does not free anyone, a company, an entrepreneur, or an artist, from these obligations to think about being like a responsible actor and speaker in the world. And so for instance, in all the stuff that I build, at the very final layer, I put in a word filter. Like none of these machines are ever that I create, even though, I mean, because I don't know what's in that Training set, that's 150 megabytes of text. I didn't read it.

You don't have people to read your monkey.

I didn't. Yeah, exactly. Exactly right. Exactly right. The monkeys, the monkeys are on the loose. And these are, these are some old sci-fi stories too, which is, it was, it was a different time. And they're actually, there is some, some—

It's like watching ads from the '70s. Like it's not appropriate.

Absolutely. Absolutely. Um, and, and knowing that, and, and I just being kind of mortified by the idea that someone trying one of these tools at home, kind of playing around with it would type something and then hit the Tab key and then see something that was like really bad, like sexist or racist or just ugly in any of a number of ways. I just found that so— to my knowledge, it hasn't happened, but just the vision of it is so mortifying. So yeah, I built in some safeguards to make sure that that won't happen.

Do you think that the safeguards are one of the most important things to be built in this space?

Yeah, I do. I do. Now that you've done it. I think it's really important. Yeah. And it's just In part because they just prevent weird stuff from emerging, weird, damaging, dangerous stuff, and also because thinking about them is a really good habit, you know. It just— so much of it, so much of the work tends to be focused on making it work, and you get— and when it just— when you see that spark of like, "Oh my god, it's writing a sentence!" Yeah, it's so exciting. And but— and so, so thinking about the safeguards makes you just kind of take those next few steps into like Okay, it works. Good. How is this gonna work in the world? Like, how is it gonna fit into a broader world, not just my little laboratory here? And I think that's important.

All right, thanks for listening. Get in touch on Twitter @Assist. DMs are open. We're super interested to hear who you think should appear on the podcast. Please check out the full interview with Robin, subscribe so you never miss an episode, and share this with someone who cares about how we make sense of these changing times.

Machine learning.

Machine learning is made by Paul Chufo and Michael Alsesser for Limina House. Have a great day.