Robin Sloan: Chapter 1: Good AI Makes Writing Harder, And That's OK

M-A-C-H-I-N-E-Y-E-S-I-S-T. It's Machine Yearning from Assist. It's a podcast where we think and dream about the future of AI, the talking internet, and how we're reshaping our culture. This chapter is an excerpt from our longer conversation with Robin Sloan. Robin is a sci-fi writer, social observer, and dedicated AI enthusiast. Robin has been bootstrapping his way into some fascinating co-writing experiments with the machine. We find inspiration in the daring he brings to his writing and the collaborative process. Lately, Robin has been talking about how with his machine learning co-writing experiments, quote, the goal is not to make writing easier, but to make it harder. Robin is funny, welcoming, and he challenged us to think about new ways to look at what's possible with human-machine interactions, which will have huge implications for creative people, the talking internet, and brands. We loved our time with him, and we think you will too. Enjoy.

Hey man, how are you?

I'm good.

Thanks for coming by.

My pleasure.

I'm going to start off, I want one question. Did you always want to be a writer?

Yeah, yeah, yeah. I was a public library kid, you know. Yeah, absolutely. Spent a lot of time in the Troy Public Library in Troy, Michigan, where I grew up. And on some level, I was the kind of kid that, like, as soon as I saw those books on the shelf, I knew I wanted to have a book with me.

Do you remember the first book you saw?

The first book? No. I do remember I was a devotee of the Choose Your Own Adventure books. There was a little wire spinner rack that had those on it. Um, and I like books of myths too, big picture books of like Greek myths.

Why?

Oh man, I don't know. They just were They were magnetic. One in particular, there's actually a whole cult around this book. It's called Dolère's Book of Greek Myths, and it's huge. It's like the size of a coffee table book, but it's for kids. Beautiful illustrations, just these total rich, colorful illustrations you can fall into, and these really great, crisp, compelling renditions of all the Greek myths. And I used to check that out from the school library, from the elementary school library. Every week. I would just go back and like check it out again. Basically, I had a lock on Greek myths. If there was any other elementary school student in my school who wanted to like learn these Greek myths, that was not available to them.

Seriously?

Yeah, for a year.

When did you go from reading to writing?

You know, pretty immediately. And that's where it ties though, from the beginning, or almost the beginning, into computers, because we had a Mac Plus in my house.

What was a Mac Plus?

The The Mac Plus, that was like—

My brother got me into Macs when they turned colors.

Yeah, yeah, this is pre-color. This is this black and white screen. This is somewhere in the middle of the '80s, maybe '86, '87. My dad brought home a Mac Plus. It was really the first computer that I had used. There probably had been some Apple computers at school or whatever, but it was the reading and the writing and the books plus that computer and like learning how to use that computer and kind of hack on that computer a little bit. A lot of my earliest creations weren't like handwritten on paper, they were typed into whatever text editor that was.

Well, that ties into now. It seems like you've always been fascinated by these writing tools.

Yeah.

Like, talk about these writing tools you've been building now. I mean, do they feel the same as then? Like your blog post I read, you're basically in a DOS text editor.

Yeah, yeah, it does, you know, it does feel the same 'cause there's that sense of like, I'm probably a little too interested in the tools sometimes, to tell you the truth, you know, to my, to my own distraction. I'll get kind of wrapped up in thinking about the tools and trying new tools and creating my own tools where probably I should be spending at least some of that time actually using them and, and creating more things. But I don't know, that's just my brain. And yeah, and so given any new technology, and I mean, this has been true of like the web, the early web, it's been true of mobile phones and like the kind of the interface of the app on the screen and now machine learning stuff. For some reason, my impulse is always like, yeah, how can I learn to write something with that or on that in like a new way?

And when did, when did it go from a new tool to like kind of the thing I'd love you to explain, like what I've read about now and what you're working on now as it relates to machine learning. Almost said machinery. Point taken. And what got you inspired by these new tools? Where did— when did it go from writing to you saw machine learning to applying that to what you're doing?

I had, like everybody, read a ton about machine learning and AI tools and all the kind of hype and self-driving cars and all that kind of stuff for years. I ran and thought it was interesting, but no more interesting than any number of other kind of like cool high-tech things that you read about in Wired or The Economist or The Atlantic or whatever. I ran across a blog post that a lot of people read and liked by this researcher named Andrej Karpathy. What does he do? You know, now I think he works at Tesla. He, at the time that he wrote the post, he was at Stanford. He was a PhD student at Stanford and had been doing all sorts of work, you know, on just all kinds of different neural networks and applications. And on top of all that, just had, and I think still has, a real gift for explaining it and kind of sharing what he knows. So he'd written this post about using recurrent neural networks to generate text. And I'm pretty sure at the time I read the post, I wouldn't even have recognized the term recurrent neural network. And I didn't. And again, it was my understanding of all this stuff was very kind of superficial. So the post was interesting. Um, and his voice is just really engaging and welcoming, you know, it's not sort of the voice of the, uh, of the engineer who's like, well, you probably won't understand this, but we'll try anyway. He was like, he was like, he was like, all right, let's, I'm going to break this down and tell you how this works.

I love how you start moving the water bottles around like they're Legos.

That's right. That's right. You could see the engineer pours his water. Yeah.

Yeah.

Yeah. Yeah. The, um, the thing, the thing that really caught me though, it wasn't the technical parts of the post or the neural network stuff. It was that he, of course, shared examples of text that these systems had generated. And the text was really weird and really beautiful. I mean, that's what it was. I saw there, like, in the shape of the text itself, something that I found really artful and kind of magnetic and interesting. If that hadn't been the case, if it had been technically interesting, but then, you know, the text it had generated had been really flat or kind of generic or blah, then I think I would have moved on to the next post and the next post and the next and never even have spent another second thinking about these tools.

Do you remember what it was that was weird and beautiful?

I don't remember from his post. I do remember there was a phrase, a terrific phrase that leapt out at me after I had started tinkering with the tools myself. And this is probably the next week. I mean, pretty quickly after that. Yeah, I was like, I should, I should try that out and see if I can, you know, get anything interesting to come out. There were a bunch of tools all kind of bundled up that were just super easy to download for a kind of enthusiast-grade programmer like me. I could just kind of download it, get a bundle of text, set it up, get it training, and within a couple days started to see some output. And there was just this one little phrase. It's not going to sound very magical, but it wrote something. It was kind of this big blob of text because it kind of goes on and on and on. Of course, these machines, they do the text so well, but on the level of the word and the phrase, it's so beautiful. But then you zoom out a little bit and you get to the level like the sentence and the paragraph, and it's basically like an insane person writing because it just—

because they don't—

because at least at the moment we're in now, they don't kind of understand these larger structures of like the idea of a story having a plot or where it's going. So who knows what else was happening in this text, but I remember it wrote, "The slow sweeping tug moved across the Emerald Harbor." And it's a tiny thing, but like, "the slow sweeping tug," that's just a cool thing to say and a cool thing to read. Like, "slow sweeping" is kind of a weird word.

Opening to some book.

Yeah, absolutely, exactly.

I don't know, some book. I don't even know who would have wrote it. Moby Dick? It's like somebody wrote "the slow sweeping tug." Totally, totally.

And that's saying a lot. That's saying a lot that it was this thing generated by this computer program that had whatever that is, that touch of art or beauty, the unexpected. So that in my own— that was maybe the second, like, stroke of the piston. First I read this blog post, then I tried this stuff myself, and I read about the slow sweeping tug moving across the Emerald Harbor. I'm like, I will pursue this.

Is the machine the artist thinking? Artistic thinking, or is it the person?

Well, it's a combination in that case, because of course, you know, in this particular instance, you're training it on text, and so there is an underlying art, which is all the art that's, you know, gone into all the books or, you know, screenplays or novels or whatever that you're kind of feeding into the system. So that's all there, but boy, there's a lot of it, and it's not like— I mean, sometimes you can do this very specific thing where you train a neural network on Shakespeare and and like learn Shakespeare. That's fine. All the experiments I've done have been much larger sort of bodies of work, so it's not just one author, one voice, it's tons and tons and tons. So it's all in there getting kind of just chopped up and stewed together. And so I actually think then when you look at the text that comes out the other end, I actually think it is appropriate to ascribe some of that, some of that beauty and some of that creativity to the to the machine itself and to the neural network. Yeah, I think it's doing something new in the same way, in the same way that frankly, I think my brain does a certain kind of work when I sit and read like hundreds and hundreds of books and then go to write something myself. I mean, I think the two processes are— it's not the same, very different levels of complexity and sophistication, but I think they're not totally different either.

What did you mean when you said the goal to everything you're doing is not to make writing easier? To make it harder.

Oh yeah, yeah, yeah. I wrote that in one of the posts that was explaining—

I read it and I was like, yeah, it was like one of those lines where it was like, wow, that makes a ton of sense, but I have no idea what you mean.

Yeah, yeah, yeah. You know, it's not to kind of retreat into analogy, but I think of other tools and other domains, like something like oil paints. Have you ever tried to paint with oil paint? I've only tried a couple times oil painting, like my house at home.

I couldn't get off my hands for weeks because I'm like, why are we using oil?

Yeah, well, so this— it's a really gnarly media, it turns out. It's really hard to work with. Nobody would— nobody ever said, "Oh, you know, I want to just paint a nice easy— I just want to have a chill afternoon of painting or image-making. Oh, I know, I'll use oil paints." They're really challenging, and it takes a lot of skill to do all those effects that we now admire, you know, on the walls of museums. People who play games sometimes talk about a skill ceiling. Like the people who are really into like really challenging video games, they like them because the difference between people who are competent and people who are really, really good is like quite a difference. Whereas there's other games where you just get good at it and that's all there is. You're good at it. There's no such thing as being very good or extremely good or world-class good. And I think the same goes for artistic tools. Like, you know, if writing in a text editor is like drawing a portrait with a pencil or a piece of charcoal, what is writing with oil paints? What is that thing that's actually more difficult and it kind of like you have to learn some new techniques, you have to learn how to handle materials, but then in return maybe you can produce something that's even more beautiful.

All right, thanks for listening. Get in touch on Twitter @Assist. DMs are open. We're super interested to hear who you think should appear on the podcast. Please check out the full interview with Robin, subscribe so you never miss an episode, and share this with someone who cares about how we make sense of these changing times.

Machine learning.

Machine learning is made by Paul Chufo and Michael Elsesser for Limina House. Have a great day.