INBOUND Bold Talks: Kathy Sierra "The Secrets of the Whisperers"

So I'm going to talk a little bit about the science of motivation and the way Laura pitched this to me incorrectly, since I didn't think there would actually be even anyone in the room is talk about something here, as opposed to my talk yesterday about something that you're just passionate about, that you think is important. So this might not go exactly where you think it will go, but this is the topic that's really important to me. So that's my horse, Dreamer, which she rode. And so I am a horse trainer. I also was a game developer and a programmer for many years. And you'll see why this matters in a minute. This is. I'm sure most of you remember reading about B.F. skinner, learning about B.F. skinner in your psychology class. This was his the Skinner box where they trained the rats through various techniques. And this is still the primary animal training technique. And I'll get to the fact where now we're using it on humans. So I'm going to give you a little understanding. I know for some of you this may be really basic about what is operant conditioning. And that's what Skinner brought to us, and that's what people are using today. That's what most gamification is, although there's more interesting forms. But this is what most of it is, even though a lot of them don't actually know this is what they're using. So. And this is where I need your help. So I'm just going to explain these four quadrants and if you understand this, you know pretty much everything about operant conditioning and animal training and hopefully some insight onto when you wouldn't necessarily want to use this on humans. So I'm going to go through these four quadrants. So now Laura is going to pretend to be my horse. So come over here. How many of you have dogs or have trained dogs or tried to train dogs? Cats? I've trained dogs and horses. Cats. I've tried to. So with + R, which is positive reinforcement, or let me back up. Operant conditioning is really just based on consequences, affecting behavior through consequences. And those consequences are either going to be negative consequences or positive consequences. And then you're either going to try to get behavior or to reduce or stop behavior. So plus R means rewards. You're trying to get behavior through the consequence of giving rewards. So that would be a really easy one. So Flora's my horse, so I'm going to try to entice her. I'm going to say, can you just come forward? Step forward, step forward. Good. And then I give Her a treat, which it's a virtual treat. Sorry, but. So that's a pretty easy one. But that's the one I'm going to come back to with a lot more detail because it's a little bit more complicated than most people think, both bad and good. This is the one that most horse trainers use. This is the one. If you've ever heard or seen Horse Whisperers, this is what they're using. It's probably 95% of all horse training is minus R. Now, minus R is called negative reinforcement, but it doesn't mean negative like it's a bad thing. It just means subtraction. It means you're taking something away to get behavior. So. And if you're parents, you've done some of this as well. So now I'm going to use negative reinforcement again. Most animals are trained. I'm just going to start, like, annoying her. Oh, good. So I stop annoying her when she does what I want her to do. So that's negative reinforcement. You just apply pressure or discomfort, and then you remove that. You subtract that when the animal gives you the behavior that you want. I will say that these two combined is most animal training of all forms of animals. Now, some animal trainers will also use punishment. Now, some will use it only if there's something like really dangerous or your life is in danger. Some use it as a technique. Some use it, you should do what I want. So some use it by. They do what's called escalating the pressure. So they take minus R negative reinforcement. They're applying pressure, which I'm going, now, don't do anything now. So don't listen to me. So if I keep annoying her, right, One approach is I'm just going to, you know, wait her out, right? And maybe I'll try to be more stimulating in some other way. But a lot of trainers will just simply say, you're not listening, you're not listening. You're disrespecting me. I said move over. And so that would be punishment for not listening. So plus P means you add something uncomfortable or painful. And between those three, you have what makes up most public school education. So think about this. And a lot of employee management relationships are based on those three things. Think about this, and we'll get to why that matters again in a minute. Now, the last one is negative P, which means you subtract something as a punishment. So a few people use this with animals, but very rarely with horses, because what it would mean. Now, I'm going to pretend you're a human, right? And you didn't do what I wanted. So hand over the iPhone. So I take something away as a punishment. Parents are really good at this. Okay, now you can go sit down. Oh, wait, no, no, no. Actually, I still need you. I still need you. I forgot about this. How many of you have used clicker training or heard of it? Okay, I'm going to show you this really quick. Huh? I thought you were good. Yeah, I thought that was good. I had the clicker because there's something interesting about positive reinforcement. If you have a dog, it's pretty easy to take the treat and the instant they give you that behavior, you're shoving the treat in their mouth. It's really hard to do that with the dolphin training. So dolphin trainers are the people who came up with this, but now horse trainers are using it a lot because also really challenging to deliver the treat at the moment. They did the behavior. And in a lot of animal studies, it's like if you wait more than three seconds after they do the behavior and then you give them the reward, they don't necessarily connect it. Right. We don't have a good language to say actually, that treat I'm giving you, it was for that thing you did over here. Squirrel. Exactly. So Laura, pretend this is a jump, right? So now I don't have any equipment on Laura and I want her to go over this jump. So. So first I'm going to actually lead you, which you already know how to do. Keep going, keep going. And now go over the jump. Stop. Now she's standing on the other side of the jump and I give her the treat. This was the only option that we had without the clicker. So the clicker gives us a chance to decouple the actual thing I'm rewarding you for and the delivery of the treat, I'm separating those in times. And I've never seen a horse take longer than maybe two 10 minute sessions to actually figure out that the thing they're doing when they hear the click is exactly what they're going to get, is exactly what they're being rewarded for. Because if I don't do that. Come back here now I'm going to stand back and I'm going to say, ok, now just go over the jump. Ok, now she was a very unintelligent horse because an intelligent horse would have said, my horses are really intelligent, would have said, well, I got the treat from standing right here with my butt to the jump. So the intelligent horse goes, I'm just going to Walk around it, because they're not connected to that. It was actually going over the jump. At the very least. Even if in the back of their mind, they're. I'm pretty sure it's the jump, but hey, I'm just going to try this. So I have to be able to click. It's pretty loud at the moment that she's actually in the air. So now try it again. Okay, go, go. Now my horse. Actually, all of my horses, but one in particular. If I click, my horses will even know which foot they're actually lifting and how high. So I have to actually teach them. No, I meant either leg is fine. Because for a few days after teaching them something, they'll start walking around like they're lame. Okay, you can go sit down. Good horse. Excellent. So there's a lot of. A lot of arguments today in. I hear this about gamification, I hear this about employee employer relationships, and of course, lots of heated battles on the Internet over animal training, which is, which parts of operant conditioning should we use? And there's lots of arguments about punishment. The problem with punishment is that Skinner found that punishment is very effective, but it also has side effects. And you might not want those side effects if it's a horse. Especially if you punish a horse, they might kill you later. They might be very, very obedient, but they're just holding it in and they'll explode later. So punishment has consequences for humans, for dogs, for everyone. So that one is often taken off the table, but the other two plus R, minus R. Do we give rewards? Do we just use pressure and release? What do we do? Are we giving the right incentives? Are we giving the right rewards? Are we rewarding them for the right thing? This is where the discussion is when they really should be talking about, should we even be using operant conditioning at all in this scenario? And that discussion is not happening nearly enough. And it's a really dangerous situation, but counterintuitive. I understand why people are taken with the idea of giving rewards and incentives. It makes perfect sense. How can incentives ever be a bad thing? And it can be a really scary bad thing. So we're missing that. It's super motivating, though, right? Here's the thing. That's my dog, Bowie. So the thing about positive reinforcement that you don't get with anything else is that. And those of you who have a dog and you've taught it even two or three tricks, you know exactly what happens, right? The dog is sitting there, you're holding a treat, and the Dog just goes, well, I'm going to go through all of my tricks. Is it. This is it? High five? Is it, you know, they roll over, they're seeking. And that's an effect that happens in the brain, and it happens in humans, it happens in dogs, happens in horses. And in many ways, it's very desirable. And they are highly motivated, highly engaged. The secret is, what are you doing with the reinforcement schedule? The secret to that behavior. And by the way, this is, of course, the secret behind slot machines. And when they added video game technology to slot machines, then, for the first time, slot machines became the number one moneymaker in casinos. Because now they tweak it, they tune it, they hire the best behavioral scientists, big data. They've got the people from Zynga, and now suddenly, they've made these things ten times more addictive. And not just to people with, this is a misconception, addictive personality. These things are designed to be addictive. And there was a time years ago when I used to think, well, isn't that what we want to do is get our users addicted to things? Now I realize that even if that actually worked, it would still have a downside for us as the people trying to get them addicted. It's not the upside that it might seem to be so. And the only thing that hasn't scared me as much when I see some of the gamification techniques is I say, but they're not really very good at it. They haven't been animal trainers. They're just not very skilled. What frightens me is now they are starting to get more skilled, because what we know is that, like when Laura was jumping over that thing, if I gave her a treat every single time, very quickly, she would just start phoning it in. But to really trigger that effect in the brain, you start decreasing the rewards. And you make that once the animal learns what you want, and then you make them intermittent. So intermittent variable rewards. There are many ways to give people rewards. The least useful is when you just give them a reward every single time. When it's unpredictable, then it becomes extremely powerful hook in their brain. And in fact, the times I have to use this, I do. Some of you may be aware of this. I mean, Skinner was able to get behavior that looked very complex. And he actually had pigeons that were trained to go in the front of the missile and do the missile guidance. Basically, they were trained to do course correction. Now, thankfully, they didn't actually have to deploy any of those pigeons, but he proved that you could teach behaviors and get Behaviors that looked extremely complex. But now, if you're using this on people, here's one thing Skinner did eventually say. It only looked complex. It was just a chain of, of very simple, very mechanical behaviors. So if you hear anyone talk about the word game mechanics, that creates mechanical behavior. But it can look very, very exciting and engaging to see people when they first start doing this, because you get that seeking behavior where they're just like, oh, I want more, I want more. What do you need me to do? I'll do it. I'll do anything. I'll try new things. But of course, has a dark side. Has a really dark side. And it's been very hard to talk about this to people because until you've seen what this science can do, again, it seems like it's all upside. How can incentives and rewards ever be bad? But it's really bad. First of all, the cognitive scientists know that it takes over your brain. And you see this happen, right? And you probably have seen it happen with animals, dogs, where they just get so excited, they're just beside themselves. It's taken over their brain. They actually stop thinking. And this happens to humans as well. They actually stop thinking because their brain is now in a completely different and primitive mode. And again, it's one that we used to think was a good thing to take advantage of, but has a dark side. This is the horse that Laura rode. If I remember correctly, you were galloping on him and you guys just went passing right on by. This horse almost died. And I will be actually writing a post that I gamified my horse almost to death, because that's exactly what happened. And I'll talk about that in a second. And we're using it on people. Now, how many of you have seen Dan Pink's TED Talk on motivation or read his book on motivation? So I urge everyone, no matter what your role is, please see his TED Talk. You don't have to read the book. It's a great book. But you will really get the gist from watching his TED Talk. So some of these studies, you'll see, for years and years and years, we've known that this is true, that there is a dark side to rewards even when they're done right. In fact, not just even when they're done right. It's worse when they're done right. And this is the dark side. In fact, it all started with monkeys. Monkeys were solving these little wooden puzzles decades ago, just tiny little wooden puzzles. And the monkeys really loved doing it. So they said, well, let's Give them a treat for every puzzle they solve. And surely their puzzle motivation should go up because that's what you see with so many other behaviors. But in this case, then the monkeys lost interest in solving the puzzles. Did a study with little kids who were drawing with markers, right? You take any little kid, you put them in a room with paper markers, they're going to start drawing. They started giving the kids little ribbons and awards for their drawings. Very quickly the kids lost interest in doing those drawings. And the drawings they did became much less than the ones they were doing without the reward. And on and on and on. If you've read anything by Teresa Amabile, she has the recent book called the Progress Principle. They did a study where they took writers and they had them in a room. They had already been pre evaluated by other whoever is qualified to evaluate the quality of writing. They had them do these writing pieces. Then they took them and they had some of those writers simply exposed to a narrative. A list of reasons why people might write. One group had a list of reasons that was mostly external rewards. It could be like the recognition. I mean, not just money, but things like recognition, peer approval, whatever it might be. The other group was simply exposed to a description of very intrinsically rewarding reasons for writing. Then you can guess what happened. Then they were asked to write. And the people who were simply exposed to the list had a much decreased quality of writing. So we have all these heated battles, but under some conditions, any of those four quadrants of operant conditioning can cause a demotivation for things that either could have been or were intrinsically rewarding pleasurable to do for their own sake. So you can imagine right now when I see people using this to encourage kids to read, give them rewards for reading. And now there have been studies about that. It's horrifying because they have potentially just sucked the chance all the life out of that experience of reading. And of course we do this with our users. This is something that I was really good at. And in fact the reason I had so many problems is because I was actually skilled at doing it. So instead of talking about that, we should be talking about whether we should ever do it. Now this, I don't know. You probably won't be able to see this very well. And that's okay. This is just something I want to encourage you to read up on. The main theory of intrinsic and extrinsic motivation right now is called self determination theory. It's the theory that's underlying what Dan Pink is talking about. Even though I don't believe he actually mentions the term, but that's. Those are his sources. So this is the leading theory of motivation and it says it's not about extrinsic and intrinsic, it's about a continuum of extrinsic rewards. And the problem is the left hand side is what we're usually using to incentivize our users. And if we use those two on the left, we already know that it has a pretty clear tested for decades, hundreds of studies, independent researchers has a clear effect on long term after a short term flurry of engagement, which is why people get so excited about it and then keep doing it. It leads to demotivation for these, for intrinsically rewarding experiences. Which is exactly what happened to my horse. He was so demotivated by how good I was at this. This particular horse started failing at neurological exams. But I watched the Dan Pink Talk, went through a complete transformation of what I did. Basically taught him how to be badass as a horse and now suddenly he's actually one of the higher ranked Icelandic horses in the country right now. So read self determination theory. But all you really need to do is look at the Dan Pink TED talk and you'll be way up the curve. And just think very carefully about incentives. There are places to use them, but there's such a counterintuitive downside that you have to be careful. Thank you very much for listening. Thank you.