The Great Hedonometer
25 Aug 2016 · 46 min
Andy Reagan built the Hedonometer, a tool that scores the emotional arc of stories by measuring sentiment across thousands of words. He analyzed 4,000 books from Project Gutenberg and found six basic emotional patterns in fiction. Writers can use it to compare genres and diagnose their own stories.
Chapters
Hello, and welcome to the Story Grid Podcast. This is a show dedicated to helping you become a better writer. I'm your host, Tim Grall, and I am a struggling writer trying to figure out how to tell a story that works. Sean Coyne is joining me soon. He is the creator of StoryGrid, the author of the book Storygrid, and he is an editor with over 25 years experience. Now, a few weeks ago, Sean mentioned this Hedonometer, this research into stories and how it's being applied to stories and how we see the same six story arcs over and over. And so we were talking about this some more and decided to invite one of the researchers on the show. So this episode is a bit of a departure from us, but it becomes really interesting as we talk about not only what this research shows, but the future implications of it. So I think you're really going to enjoy it. It was a lot of fun to talk to this researcher and dive into the work that they're doing. So let's jump in and get started.
So in this episode, we're diving deep into the research that we mentioned last week that Sean mentioned last week with the Hedonometer. And we have on now with us Andy Reagan, who is a part of this. So, Andy, I just want you to tell us a little bit about who you are and what you do and, you know, specifically the work you're doing with this hedonometer. Research.
Yeah. Great. So, yeah, my name's Andy, as Tim said, I'm a PhD candidate at the University of Vermont up here in Burlington, Vermont, and I work with the Computing Computational Story Lab research group. So, you know, stories have been in our framing for a while. The leaders of the group are Chris Stanforth and Peter Dodds, professors here at uvm, and we're affiliated with the Complex Systems center here. So complex systems are, you know, studying all sorts of behaviors and data systems that are out in the real world. So we in particular have studied the Hedonometer. So we've developed this tool which allows us to measure happiness, and we've applied it to a lot of different areas. So if you go to our website, for example, hedonamineer.org, you'll see the first thing you'll see is a time series of Twitter. And, you know, there's a lot of interesting patterns that you can pick up from there. Most recently, we've gone and looked at stories which I think you guys heard about last week. And, yeah, we're going to dive more into that.
Well, Andy, first of all, thank you so much for being here. This is Sean. And needless to say, when I read about all of your research, I was just so over the moon because, you know, one of the things that I've been doing as a editor in the major publishing houses and in my personal career over 25 years is analyzing stories in a very similar. Without the data, I have no ability to use a computer or know about algorithms. So what I've been going on is through my own intuitive sense of how stories are moving positively or negatively from, you know, the beginning to the end. And so one of the things that really kind of fascinates me is sort of this evolution of this entire realm called, I think it's called the digital humanities, which seems to be sort of the beginning of a means to analyze literature and art through intense machine algorithms and attaching specific kinds of sentiments to language itself. Is that correct? Is that what sentiment analysis is all about?
Right. And so if you go all the way back to, you know, the original natural language processing people were by hand making concordance. Right. And so now that computers have the computing powers increased, I can run books through on my laptop. The digital humanities has kind of come about where we're taking, we're going beyond that just counting words, which was a concordance. You know, you'd go through and count the words, and then real literary scholars could take those word counts and take that data and use it to better understand the text. Now with what we're able to do is we're going to be able to go beyond counting words. And we're starting to investigate some of the theories of literature using natural language processing on its own.
I get it, I get it. So, you know, the other thing that struck me so hit me on top of the head was this notion of these six core story arcs. And when I say arc, what I'm talking about is the emotional movement of the lead character throughout the entire story or the small cast of characters in a multi character story. And how you guys. And I think there's some even more research being done by. I think there's somebody in University of Nebraska named Matthew Jockers who's doing similar work to you. He's using sentiment analysis and I think he's coming out with even a book in September called the best seller Code, which essentially is going from word to word to word, but in larger chunks. So you're not exactly parsing every exact paragraph, but you're sort of taking pictures of individual stories and then tracking those. Is that correct?
Right. You hit on a bunch of different things. So we came up with six stories. There's been a lot of theories over time that have proposed different kind of framings for how narratives work, and those have enumerated different amounts of stories. You guys talk a lot about Campbell, I think, and he came up with the monomyth. Right. So that was the idea that there is one story and, you know, variations on that story do create some differences. But if you whittle away, you can get to these core elements of every story. And there have been different theories. Time. So one of those is three plots. Right. So this is Foster Harris back in 1959 saying that there are three basic patterns of plot, kind of all from a central pattern of conflict. And even more recently, Christopher Booker published a big volume representing three decades of work, and he boiled it down to six. Yeah, seven. Yeah, he says the seven basic plots, but I'm not sure comedy is a plot.
Yeah, I had a problem with that, too. But this is nerd talk, so. So we'll leave that to another time. But keep going.
Right. And so there's a lot of. You can keep going. There's people that have found 20 and 36. And so now we're beginning to take these theories where people read lots of books. I assume Christopher Booker read hundreds of books and was able to whittle this down. Now, with a computer, we can read many thousands. So in our analysis, we're able to look at all of Project Gutenberg, which is a collection of 50,000 ebooks. We only look at a subset of those because we're actually analyzing fiction. So we look at up to 4,000 books. But, yeah, we were. We're able to answer these original questions using the data.
All right, so let's just try and delineate. I'm going to try and have a conversation with you about some of my theories and how they merge and ebb with what the six central plots are. So can I cut in? I'm going to cut in, Tim.
I know you're so excited. Sean, let me cut in.
I know.
Yeah.
So, Andy, I want to back up. And when you say that you measure the happiness.
What does that mean? How do you go about doing that?
Yeah, and that was the second part of what Sean just asked was about the methodology. And, you know, some of the work that jockers have done, there's some differences in methodology. And also what we think the methodology is doing that I think are definitely worth highlighting. So what we're doing, lecture center, is we're looking at individual words. So, you know, you open. You open a novel. And if you were to look at the happiness of individual words out of context, we have scores for individual words. We have scores for the 10,000 most used words in English, which will cover about all of the words, 90% coverage of a text. And so we go into the book and we look at those individual words and replace them with their average score. So each word we had scored by 50 people on happiness, this is the particular sentiment dictionary we use. There are others. And so we had a crowdsourced methodology. We had 50 people rate each word. And so we go through a book and we replace each word with. With its score. So what you end up with is a book that's just a bunch of numbers. And for us to begin to take that into the overall sentiment of a piece of text, we'll average the scores of those words. So what we're doing in the particular choice we made was to go through and then take big chunks of text, so 10,000 words at a time, and average the sentiment of all the words in those 10,000 word chunks. And then we take that 10,000 word window and we nudge it a little bit. So we take a few off at the start, add a few more on, and generate another average score. And then we do that throughout the book until we have 200 points. So we move it just enough so that we end up with 200 sentiment points throughout the book. And, you know, there are other ways you go about doing this. So in particular, the way that Matt Joggers goes about doing this is measuring individual sentences. And so what he'll do is take kind of the same approach, but do it to a sentence. So you get a score for a sentence, and you get a score for the next sentence. And then what he'll do, because the sentiment, the sentences actually end up being really noisy. You get a happy sentence, sad sentence. Happy sentence. Sad sentence is taking an app, a rolling average, and then from that, doing some more analysis. So originally I think he was doing that and then taking that noisy time series, which is kind of up, down, up, down, and then doing the Fourier transform. But that doesn't work as well because it's very noisy and you get some unintended effects. So from there, taking the averaging window, it kind of smooths out the contributions of each word.
So you're basically just walking through the book and taking like big chunks at a time and saying. And so you just have certain words, like this group of words. If these show up, that weights to happiness. And if this group of words Shows up, it weights to sadness. And that's kind of where you get these valleys and troughs.
Yeah.
So the happiest three words in our set are happiness. The happiest word is laughter. And then love, and then happiness. So if we see. If we're seeing a lot of mentions of love and happiness, that's going to bring the score for that section of the book up. Status words we see include, you know, terrorist and death. If you start seeing a lot of words like that, it's going to just bring that average score down. I think it's kind of important to emphasize, too, these methods are getting better at rating individual sentences, but they're not really good at that individual sentence level. So, you know, if you see the word death in a sentence, it's going to rate that sentence negatively, regardless of what else was going on around. Could just be, you know, you're analyzing Harry Potter and you saw a Death Eater. Right. They may not mean that something negative was happening. You know, it could have been going away. But at an individual sentence scale, it's hard because you're missing that context. So we'll kind of, you know, smooth that out by looking at a lot of text.
Well, yeah, I was reading that, you know, Jockers has gotten some criticism for that very thing, I think a woman named Anne Swafford. Swafford, yeah. Yeah. And I think that's a very valid point. Now, I just want to. I've got a million things that I could talk about in terms of the five commandments of storytelling that I talk about a lot in StoryGrid and how they could be applied to your system. But I want to sort of go Philip K. Dick for a minute here and.
Think about, you know, what's the end game here? Where does the analysis of story lead, in your opinion? I know you can't say one way or the other. You know, storytelling is such. And I talk about this all the time, such an innately human characteristic. And many people, myself included, believe that in the concept that Carl Jung sort of came up with is the inner genius. Not Carl Jung, but he took the Greeks and the Romans. Sense of each person being sort of born with an inner genius in which has to sort of come out and be realized for that person to pursue their own personal destiny. And Campbell talks about this, too. So how do we get exc about sort of analyzing stories from an algorithmic, computational data mining point of view? Is there going to be a moment when we could pre program some kind of robot that would play with the language in A way that would give us these ups and down curves of positive and negative emotional valence. Do you think that there would be a time, and I don't know that Kurt Vonnegut ever believed that this was true. I thought what he was always saying was that, you know, it's only a matter of time where the computers will be able to prove my theory. And they did. I think he was absolutely correct. And he was saying back in 1994, whenever it was in that, that lecture that's on YouTube, that it's only a matter of time before data mining can bring this together. And I think you guys have really shown that that's true. But my question, back to my question is, do you think there's a time in the future or near future where some sort of robotic algorithm could create a story that would be similar to the Da Vinci Code or any of the other sort of pulp fun stories that are around today?
Right. So you hit on a whole bunch of things. I think that it's, you know, where we are now is further than we were. You know, we went back, you know, started talking about concordances and counting words. And the methods that we use for natural language processing and sentiment analysis have gotten a lot better. Right. We're now able to go into each book and look at the happiness of the words and do a rolling average and on top of that, do analysis to see which are the most common arcs. So we were able to, I think, answer the question of Vonnegut pretty well. But I think that the methods are still pretty simplistic. So like we just described, we're looking at words. And so I think one of the important distinctions is talking about plot. You know, Booker talks about the seven plots. We're not really measuring plot. What we're measuring is kind of the emotional experience of which is created by the plot. You've got the plot as a sequence of events, and then you've got the structure, which is how those are presented. And on top of that is kind of where you get this emotional experience that's created. So I think that it's important to note that what we're really measuring using these sentiment methods is we're not measuring the plot that's underlying the story. That said, I do think that for computers to understand us and to be more human, like you said, we're innate storytellers. And so artificial intelligence has gotten better too. Right? You can ask Siri a lot of complex questions, and that has improved through these natural language tools. And I do think that one of the next leaps forward will be computers understanding stories in the way that we do. Because there was Patrick Winston at MIT put forth this strong story hypothesis. And his idea was that what really separates our mental abilities is the fact that we use stories to generalize and to understand the world. And that leaped from this is happy, this is sad. To building a more complex story is going to be a big step forward in artificial intelligence and our understanding.
Well, I'm looking at it from the other direction of we have these kind of six basic stories and they all kind of follow these arcs. You know, I'm coming from it from the standpoint of I'm trying to learn how to write a great story. And, you know, I'm, you know, struggling through how to arrange everything in a way so that it does. I mean, to me, the first thing that pops in my head with this is at the end of like, you know, when I finish my novel, I should be able to run it through the hedonometer and my book should fall in one of those graphs. And if they don't, I can probably pinpoint some places I need to go back and fix my story.
Do you think that's feasible?
Yeah. And actually, one of the things that we're working on is the ability. We're working on a web app and a desktop tool for people to download and run their own story through. So you'll get out that sentiment arc if you see that things are moving wildly from positive to negative at the beginning, you know that you're not conforming to one of those story arcs.
Well, I think, Andy, if I could. I think you really made a great point here that sort of got glossed over. And your point was the hedonometer and the research that is currently very convincing in my opinion, has delineated story, emotional story arcs. They have not delineated the ups and downs and curves. Like if you, if you were to take a look at my story grid for the Silence of the Lambs and you were to compress all the points down, it would look similar to something akin to man in whole, man in whole Cinderella tragedy, which, as you guys had pointed out in your paper, that seems to be the most popular sequence of emotional arcs in a long form novel. Now, what that means is that, and this is really what excited me when I read this stuff is that the pinpointing of the emotional arcs for me represents sequences in a novel. Now, I know a lot of people haven't read the story grid, but there are units of story that I talk about in the story Grid. And so to me, the story, the six story arc, are representative of a way to construct your sequences. And Tim, I was trying to get. Get into this in our last conversation, and it was, it got very muddy very quickly, and I fear it's getting muddy now. But my point is, is that you, the computational Story lab is not in the business as of today of analyzing a crime story. They're not going to tell you and they're not going to be able to explain to you the rise and falls of a particular external genre that we all know and love. What they will be able to tell you is whether or not your emotional arc is working in a way that will engage the reader's interest in a very fluid way. So what they are very exciting to me is being able to take a look at whether or not your narrative drive will drive people to continue to read. And so what you have been able to pinpoint are the emotional arcs that get people turning pages. So, you know, the man in the whole story is the very simple thing where somebody is neither positive nor negative in their life, maybe a little bit more positive, because nobody likes anybody who's depressed, as Vonnegut said, who's walking down the street. He falls in a hole, and he has to get out of the hole. So we want to know, as a reader or as a viewer, is that guy going to be able to get out of the hole? And what are the progressive complications that he must undergo in order to ultimately go from a fall to a rise? So these six story arcs are wonderful tools when you are considering, you know, the internal movements of your protagonist. And I love the idea of being able to. People being able to plug their manuscripts into the hedonometer because they are going to see how their novel moves sequentially. So at the beginning, they may see man in a Hole. It starts positively dips below the negative, and then it rises just a little bit above where the character started at the beginning. And then we get another man in the hole and he falls again and he rises a little bit more. That's called progressively complicating the story. And then the Cinderella, which is a rise, a fall and a rise, and that it ends with a fall. And if you look at the Silence of the Lambs, that's kind of what you're going to find. And I found, you know, when I was reading all your research and everything and Jocker's research too, and I don't, I don't think we should discount his work in any way. I think what he's Yeah, I think he's, you know, he's trying to go down into nano, going down to the word. And I think probably the next step would be for the science to understand the principles of, you know, the five commandments of storytelling, which Tim and I talk about all the time, which are, you know, the inciting incident, is it positive or negative progressive complications? Are those positive or negative progressive complications in a particular scene? So I think if you were able to have a combo plate between. You know, the other thing that I want to ask you in a second, Andy, is how you got into this to begin with. But if you were to have a combo plate between the mind of an editor and the mind, or a writer or a storyteller or a screenwriter or a radio personality and a mathematician, and for those people to bring the peanut butter and the chocolate together, you could come up with something really fascinating. But back to my question about how you got into this predicament yourself, Andy. What drove you to get into applied mathematics? And how did you get into the storytelling element of it?
Right. Well, we're interested in stories kind of as a founding principle, because all the reasons that we've talked about. Right. Stories are. They're fundamental. And I think that great goal, a goal of science, a great achievement, will be a rich understanding of the landscape of human stories. I think that that basic drive and the fact that the way in which we use stories and how important they are to us was really what drove us to get into this.
What about you personally, Andy? I mean, where did you grow up? Did you always love math? Were you afraid of stories? Because I started out as a scientist, and then I flipped over to storytelling.
Yeah. So I've always been a math guy. I have not been much of a writer or a professional storyteller at all. So I went to Virginia Tech for my undergrad, and I studied pure math, and then I came up here to work on my master's, and I actually began studying weather and climate models. So I looked at data assimilation algorithms, which are useful to combine observations and weather models to make good forecasts. And just that basic training in linear algebra and mathematics makes a lot of these algorithms, which process language possible. And so the hedonometer project was already going when I got to the story lab, and I came on board to kind of pursue this next goal, which was to get to stories. Yeah.
So what is the next step for the story lab? Are you continuing on this path? Are you trying to get a little bit more granular? What's the next thing?
Yeah, I Think that there's a lot of ways that we can directly improve the work that we've done. And we've already improved it a little bit just from some of the feedback that we've gotten. So in particular we've improved the corpus that we're looking at. Corpus of books. I think that going down the sense of is a really great goal right now. The dictionary based sentiment methods aren't that good at it. But I think that going down to getting at that granular level is really what we want to do. And we want to do it in a way, like you said, that respects kind of these commandments of storytelling and that understand that there's more than just positivity and negativity and that positive and negative language interacts with kind of the events and characters that are in a story in different ways. So I think what's going to be really important for us is to get out characters. So who is the main character and how are they being talked about? There's been some work to extract different character types across lots of books in the digital humanities. And what we're going to try to do is what I think is going to be interesting is connecting characters and events. So you've got the first thing that happens in a story, which is something that happens around the main character. But then how that event connects to the next event and what characters are involved in that progression, I think is going to be an important thing to understand. So setting that up to kind of fit into a framework that is more in tune with the way that we understand stories I think is going to be a big next step. So you might want to ask the computer a question. Here's a sequence of events and you can think of a simple story like think of Macbeth. And so you want to say, is this a story of revenge? You know, you can see that in the beginning, you know, Macbeth killed Macduff. Right. And how did that progress into what ultimately happened? There's some hidden parts which aren't in the sentiment about how which people are affected by certain actions. So if you can take the events that happen there and then connect them in a way that you can draw causal inferences. So you know, what things in a book fit into our societal norms of what's appropriate action? When would it be appropriate to take, you know, do an action as a revenge? Or if you're reading a book in a different culture, you know, that doesn't happen. That's not part of the cultural narrative. So I think getting in a model that can Start to answer those questions. And kind of framing the analysis around characters and the events that happen between them will be a really big step forward.
Is there anything in this? Because one of the things that I'm learning working with Sean is just how many times my thought, the first thought I have for a scene or a sequence is cliche. You know, I've seen it a hundred times. I was just talking with another writer this weekend and he. He like described this opening scene and I'm like, dude, I've seen that like eight times in a movie in the last two years. You know, and so I, you know, I was kind of. It's so funny because people are starting to ask my advice because I've been talking to Sean so much and I'm like, this is not as good as the real thing. But, you know, I was like kind of coaching him through it. And so I wonder if there's something in here too, of like, would we be able to run 20 books from the genre we're trying to write in and see what's happening at each of these places as a way to just avoid common cliches that show up over and over.
Right. So if everyone begins with a man and a whole story type, that tends to work. But it also might be something that you want to avoid because everyone's just seen it before.
Well, it's not exactly that. It's. It's the way the man gets in the hole. Like a man getting in the hole can be done. And Tim and I talked about this in terms of stranger knocks on the door. That is a setup for a man in a hole problem. Right? The stranger knocks on the door and says, hey, you won the lottery. But the problem is that in order to collect the lottery, you have to go do this horribly difficult thing. So what I think is really interesting here is the concept of taking this to the next level. And also, it's not only the micro look at an individual story. It's, as Tim says, sort of identifying plot point cliches that have been worked on over and over. In particular, one of the things that I was fascinated by is that I saw it was either your research or Jocker's research, where they did a global analysis of all of 19th century literature to find out whether or not what the cultural shift and the cultural sort of norms of that time period were. And as an editor and as an acquiring editor, when I was at the major publishing houses, this is sort of what editors try and do. What they try and do is anticipate the Next big thing. And one of the ways in which you do that, for example, in the early 1990s, there was the rise of what you call the legal thriller with the Firm by John Grisham. And the legal thriller became a huge, least successful and wonderful, wonderfully profitable genre of book. That to me, I thought to myself, well, why is the legal thriller becoming so popular right now? What's going on in the culture that is making people want to fantasize about the actions in the legal justice system? And those are the kinds of questions that I think you having the ability to analyze 4,000 works, you know, within. I'm sure it only took. I don't even. How long does it take to run a story through the hedonometer?
Yeah, a few seconds.
Oh, my God.
Not long.
That's crazy.
You know how long it took me to run the Silence of the Lambs? A good two years, but. Wow. A few seconds. That's funny. Wow.
Yeah, we were just looking for words. It's simple. I think doing more complex dives into it will take more time, but. Yeah.
So what do you see?
Because we asked a little bit about artificial intelligence, but I'm more interested in what is this going to do for writers to have these kind of tools now, with what you have now and then over the next five and 10 years, how is this going to help writers tell better stories?
Right. And that's almost a question that I would want to ask you. We've been able to get these things out, and I want to know, how would you use this in your work? You could take a story that you have and run it through and see how it compares to, like you said, other things in your genre. You talked about these cliche ways to get into different story patterns so you could see, okay, I want the story pattern, but I want to get into it in a different way than what's done before. I don't know if you guys use the word tropes, but this is kind of a another word used to frame the common elements that make those twists. So, yeah, I think that if you could use it to compare your work to what's coming out now and what's popular as well. And I think this fits into what Jockers is looking at now. So he is looking at bestsellers in the New York Times and saying, okay, what makes these books different than other books in terms of their emotional arc?
Yeah, like, things that come to mind for me would be, like, one is, if you've run all of the thousands of books through, this could I separate them into different genres. Where I want to see the 20 legal thrillers that you've run through, and then I could see those patterns laid out on top of each other. And what would be really cool is to dive into the five scenes that are around that peak in each book, you know, because right now, like, the idea of the work that goes into. If I want to write in a certain genre and I want to make sure I'm not hitting all the cliches, I got to go back and like pull out the 10 books I've read, 10 more books I haven't read, read them all from start to finish. And really I should be mapping them out scene by scene so I can kind of build my own art to make sure that I'm following the same path but not doing it in the same way and not doing it in a cliched way. It would be really cool to add a couple clicks of a button to be able to dive in and compare 10 or 20 books at the same time.
Yeah, and I think you could do that. And then on top of that, some light annotation would help you kind of understand, okay, well, these are words people using. This is what was really going on. I think you said that these. There's these different characteristic patterns. And talking about genre, I think what would be interesting is, so legal thrillers became a big deal. Are they following just the same patterns that other stories were before them, but just with a new genre? What is it that makes a legal thriller that genre? What would be the most predictive feature? Maybe the arcs are the same, but it's just the characters and the language is different because of the setting.
Well, that's absolutely true. Sort of the way I delineate it, Andy, is that you have an external genre and you have an internal genre. And the six story arcs track the global sort of emotional valence of the reading experience. And usually what that extrapolates to is the movement of the lead character, or in the case of a multi character story, the movement of the group of characters.
Right. And that's a core assumption of what we're doing. We're assuming that the language in the book is mostly about the main character.
Exactly, exactly. Exactly. So if you were to plug in, say, the Firm Presumed Innocent. Can't think of any other legal thrillers off the top of my head, Robert Tannenbaum's work, and you were to come up with the top five. One of the other things that I talk about are conventions and obligatory scenes, and they sort of are similar to what you're talking about when you say tropes. And so to be able to see the emotional movements of those four or five or 10 or 20 legal thrillers is going to give you the sensibility of, oh, that's when I, you know, if I really, I think if they would all sort of map out quite similarly and that you would be able to pinpoint knowing your genres, conventions and obligatory scenes in a way that I always recommend people do, you would be able to, you know, pinpoint that moment. Oh wait, that's 35,000 word in the firm is the exact, is very close to the 36,000 word. And as Tim said, you know, the writer, hard working, practical writer today would be able to go and find that word count, go to their manuscript, find the scene and say, oh, there's the all is lost moment. So that is the way John Grisham solved that all is lost moment for my lead character by using an external circumstance like wife finds out husbands had a secret affair.
That is a way of using your research to track the emotional arcs of the stories that you are most. I mean if we were to say, if we were to open up some kind of software, storygrid software, and we had the Hedominator as a tool, we would recommend, hey, first thing you want to do is pop your top 10 of your genre into the hedometer, find out their emotional arcs and then go scene by scene and discover what's in common, what's not in common, what ways can I innovate? If somebody has used the same kind of scene as a cliche to propel the plot forward, how can I change it? That kind of thing.
Now I was going to ask like if I did, I have a book I wanted to run through it, how would I even get it in there? Because there's DRM on my Kindle titles and that kind of thing. Is there a way right now to get a book, take any book and run it through?
Well, yeah, so we used Gutenberg, project Gutenberg, which is all post copyright. So there's no DRM that we had to break to analyze it. So you can analyze the books and look at them yourself, but if you break the DRM to analyze them, of course you can do this with software like Calibre, then you don't want to share that you're using it for your internal use. But that's not going to be something that I would give you.
If you want to send your story to me in just the full text form, I could run it through could run it through and see what the arcs are so far. But, yeah, I think you're going to need the full text to do that. If you own the book, you can use it for your own internal use. But it does get into some HathiTrust copyright concerns if you're using this work to share.
That makes sense. Well, Tim, I think it might be a fun project, knock on wood, in however many months, to throw yours through there and see what it looks like. See if all the advice that I'm giving you is actually worthwhile or if I'm just talking on the side of my. Well.
No, that's. No, it's just. It's interesting to see, like, I'm trying to think. So Hugh Howey wrote this short story one time. I think it was called the Plagiarist. And basically they had created all of these artificial intelligence worlds where all these people were living in these AI worlds, didn't know they were an AI, like the Matrix or whatever. And what people would do is they would mine these artificial intelligence places to find, like, books and works of art that they would then republish in their own world and make a bunch of money on. So they. Because it was legal to plagiarize computers. And then, of course, well, I won't ruin the ending, but so, you know, I was just. It's always been kind of stuck in my head of like, this idea that you could create this AI that's creating these works of art that you could then steal and use yourself. But, you know, I just think of these things as though, like, those first step towards doing these things that have been manual for so long that we then can use a computer to go faster on. And we've even had this discussion of, like, you know, of course, you know, using a word processor is much faster than writing out your book longhand. But at the same time, we've talked about how, like Malcolm Gladwell, when he does an interview, he'll transcribe it himself and then print it off and then go through it by hand and do all of this stuff by hand that he could do automatically or pay somebody $20 to do. He does it himself. And so it's just interesting, as these new tools come online, of what pieces are going to help us write better but faster, and which ones, if we try to use them, will actually pull out the hard work that it takes to create an actual great piece of writing.
I think that what you touched on is really important because what is it that really makes a story compelling? There's the arc, there's having characters that people are drawn to. I'm not the expert here. I think knowing what those pieces are and being able to extract those to create a new story would be the most important part for automatically generating stories. So I would say, okay, here's 30 John Grisham novels. Can I just make another one based on the way he uses language? And what else is going to be important? It's easy right now to train a recursive neural network, one of these language models, to say, okay, here's a bunch of text. Give me a bunch of text that looks exactly like it. And it would do that, but the story would make no sense. Sentences would be like, juxtaposed that aren't even related.
To Trump's speech.
They call it a word salad.
Yeah, yeah, yeah.
Well, I mean, I'm glad. I mean, it's kind of funny that you brought that up, Tim. But we have to remember that storytelling is the way we change people's minds.
Right?
So say you're the CEO of a major manufacturing company and you need to sell a product. I could foresee somebody who's writing the speeches for somebody like that using the Dominator. Dominator sounds like the Terminator now, but using the machine to track the emotional valences from that speech so that they can see whether or not they're hitting, you know, like some of our great public speakers, like Tony Robbins. I mean, there's a guy who's using these emotional arcs, these six storytelling primal arcs, in a way that motivates people to change their lives. And what I'm thinking as another fantastical use of this machine would be for businesses. Right. Sentiment analysis itself is. It's not just confined to looking at novels. I mean, there's big businesses that are going through the World Wide web and, you know, blog pages and, and saying. Giving reports to large corporations about how their particular brand is trending. Right.
I would suspect that there are.
Right. So, yeah, there definitely are.
I can believe you.
I mean, that, that is how, you know, we're convincing people to buy things. And so, you know, the scary sort of Big Brother esque element of this thing is using a machine to manipulate the consumer base in such a way that they're. You know, we laugh about the old 1950s subliminal seduction of the 1950s advertising age, but this is a literal. This is literal neurally manipulating people to purchase products, services, et cetera. Almost. I mean, it's almost in a way, brainwashing by using. I mean, there's a Reason why Aristotle, or was it Plato who warned, you know, the first thing you want to do if you want to control society is kill all the storytellers because they're the ones who can change people's minds in a way that can change, you know, the power structure. You know, this is kind of fun to talk about, but it's also, you know, everything. Everything has a shadow element to it, too. Have you guys thought about that in any way? Or is it, you know, is it. Is it. I suspect it's more fun to just sort of dive into the science and let, you know, these sort of esoteric ethical considerations sort of sort themselves out themselves.
Yeah, so we always tend to err on the side of, you know, doing the science and publishing openly. That's what we've done so far, and it's been a lot of fun. You mentioned kind of the propaganda side of this. And there was a. There was a DARPA grant where a few years ago called Narrative Networks. And they wanted to understand. They wanted to understand exactly this. Narratives are useful for influencing people and how can we measure those? So I think they had people hooked up in an MRI machine watching movies, but I'm not exactly sure. And you also mentioned marketers in business. These are the great storytellers, and they understand what drives people. And the other interesting thing is you mentioned Trump, the marketing and the storytelling come together a little here. His slogan is Make America Great Again. And we're talking about the story arcs. There's a story arc even in that. And so my advisor Peter is the one who elaborated this one for me. But there's the whole man and the whole story right in there. Right. And there's kind of a notion of action. So Make America Great Again is saying something about the past. Right. That things were great. It's saying something about the present. Things aren't great. And it's saying something about the future. They're going to get better again. So this is a classic man in the whole story that's encompassed in just four words. As far as the story goes, it's really well done.
I would absolutely agree with that. And it feeds into an emotional element that, you know, I talk about the beginning hook, the middle build and the ending payoff of a particular story and the fact that you just delineated that exact same thing in this political slogan. See, the way I would err would be to educate people. Right? To educate them into the forms. If more people understood the structural form of storytelling, the better prepared they will be not to become susceptible to propaganda. So somebody who knows that somebody's a fraud, they can recognize each other. A fraud recognizes another fraud. So if you're capable of understanding the story principles and saying, huh, that's an interesting slogan. Make America great again. What does that exactly mean? What is he trying to sell me using that language? And I think we all walk through our lives in so many hazes and digital hazes that we stop. You know, now everybody's talking about the power of video and how video is really going to be the next iteration of the World Wide Web and how, you know, how language and everything is really taking a back seat. Well, I think that's ridiculous because language is where it begins. Language is where the imagery comes from. So to understand storytelling and understanding that all story begins with language and with internal processes and understanding the arcs, the more people will understand these six arcs that are very simple and primal, the better they'll be able to recognize them when they're. When they're being given a load of bullshit, you know, because when somebody's manipulating you and you. You can sense it, but you're not quite sure. I know the way. I can. I can pinpoint it as they go, oh, I know the story they're telling. They're mimicking a story that they know about me, that they've read from some source online in order me to get to buy something. It's like a fortune teller. You go into a fortune teller, they look at your suit, they look at your fingernails, they look at your haircut, and they're brilliant storytellers. And they can look at you and say, oh, I can see that you've lost somebody dear in your life recently. And you go, oh, my God, how did they know that? And that is how propagandists and everybody work. So I think your work is really, really important because it educates people to these primal forces that we really need to understand. And the storytelling arcs are all about that.
Right, I agree. Yeah.
So, Andy, tell us a little bit about what's in the near future with your work and where people can go to make sure they're getting updates on everything that you're doing.
Yeah, so we have a website, heatonometer.org, which I think I mentioned at the beginning. And if you go there, you'll see the Twitter Time Series. And if you poke around top Bar, you can make it down to look at the Time Series for books. We also have some movie scripts up there. We're working on developing a tool to allow people to put in their own works and maybe you could put in on your own some works of books that you're trying to be in the same genre as and see where you line up. We're working on building out a desktop and a web version of that tool. We have a link for an email list out there. You can sign up for that that we'll send out once we have something working.
Okay. And I'll put all of that in the show notes as well so people can link straight from there at our website. Well, this is great. Thanks for coming on and talking about this. Sean. Did you have anything you wanted to ask before we wrapped up?
No, I don't think so. I think again, Andy, I can't thank you enough for being here. I'm really excited about the work you're doing and I think it's going to be very applicable for all amateur and professional writers from here on out.
Yeah, thanks a lot. And I happily take the advice from you guys on how to do that. Tim, I'm sorry if I took away from time that you guys are working on your story as well.
Thanks for listening to this episode of the StoryGrid podcast. To find the links to the hedonometer and everything we discussed in this episode, you can find that@storygrid.com podcast also at that URL is all our past episodes and all the past show notes. So if you need to refer back to anything and it's all there. If you want to keep up with everything StoryGrid related, you can do that by going to storygrid.com and signing up for the email newsletter. If you want to reach out to us, provide us feedback, ask questions for future episodes, you can do all of that on Twitter torygrid. And as always, if you could leave us a rating or review on itunes, that would be fantastic. So thanks for listening and we will see you next week.