How Does Duolingo Get Us To Learn a Language?

Behavior change is hard. Actively changing another person's behavior is even harder. But there are plenty of companies out there that have been able to crack that code. They understand that people don't always or even rarely act rationally.

I'm Kristen Berman, co founder of Irrational Labs. Over the years, we've helped countless companies understand and leverage behavioral science to increase.

The health, wealth and happiness of their users. The idea behind the Science of Change.

Was to create a podcast that demystifies the psychology of decision making.

So here's how it works. Each week I get to chat with.

Visionary product and organizational leaders to uncover their insights about user behavior and how.

They'Ve managed to change it. I match this up with some of.

The latest research in the field to get to the bottom of our core psychologies.

So let's dive in and unlock the.

Secrets behind the Science of Change. The Science of Change podcast is presented by Set Sail, World's number one revenue execution platform. Learning a language takes a lot of work and a serious ongoing commitment. It's hard, but as challenging as it is to learn a language, it's equally as hard to get someone to learn a language. One of the best products out there that is actually making it easier is Duolingo, and they're a treasure trove of real life behavioral science applications. Which is why I'm so psyched about my guest today, Duolingo's head of product management, Jem Cansu. He takes me through what they have learned, like signup flow tactics, how to help people recover from a broken streak and their KPIs for habit formation. Duolingo's job is to keep us engaged. How do they do it? They basically use all the tricks in the gamification grab bag. Things like streaks, points, levels and leaderboards. And by the way, many behavioral scientists, myself included, have been known to throw shade at the idea of gamification. It's often used as a quick fix to cover up a core issue. Let's just give someone points and that will fix the problem. But when your users are more focused on doing something just to get points than they are the actual learning and retaining, then they are missing the real point. And I do think Duolingo is likely guilty of this too. But the reality is, sometimes it works. We all need a little extra oomph and gem gives us some great insight on Duolingo's approach to keeping their users motivated.

Duolingo's core hypothesis from the beginning has always been if we make language learning more fun, we'll take users further into their journey of fluency. And it takes a lot of hours to actually go from if you don't know anything in a language to getting to complete fluency. The best way to do that, we gotta figure out how to make it fun. And quite frankly, we've learned a lot from games and integrated a lot of game mechanics into Duolingo. Duolingo is generally famous, is for a mechanic that we call the streak, for example. And the streak is what you would expect is the number of consecutive days that you've done a lesson on Duolingo. It's kind of a big point of pride for a lot of our users. Some of our users have 2,000 day streaks, which is, if you think about it, it's a lot of time invested into Duolingo. But we display the streak prominently, we make it very fun looking. So I actually just hit my 500 day streak, which I'm very proud of. But the streak is one in general, the hypothesis how do we gamify Duolingo so it's more fun? Streak is one. Another one is leaderboards. So competition is innate in all of us. If you see your name ranked on a leaderboard, you. You start feeling the competitive pull. So we have a pretty prominent leaderboard mechanic. And you see all of our design elements also have one form or another a gamification. We have a lot of progress bars because filling progress bars is very motivating. So all of our design and product principles have this layer of making it fun that's one. Another one is making it easy and easy to access. So that's why Duolingo is free. So if you look at the language learning space before Duolingo and even to today, a lot of the products you'll find you need to pay upfront. And that's what we wanted to change. We wanted to provide access to good language education. And that's why we made the whole product free. And the monetization piece we talk about is not the content. You just get added features with Duolingo plus, but everything you can access for free. So another hypothesis we fundamentally believe in is making it easy to access.

By the way, these are two of the most important principles of behavioral science. Reducing barriers by making things easy because people take the path of least resistance and increased benefits because we all have present bias. Look, learning a language is not only difficult, but it also takes time. Normally we'd assume if you're going to learn a language as an adult, you need a high amount of willpower to get you through hours of painful practice. Willpower At a high level is the ability to delay gratification. Duolingo is trying to remove the need for willpower by giving you immediate benefits for small actions. Having both of these elements work in tandem, reducing barriers and increasing benefits is typically the recipe for behavior change.

I'd love to go back and dive into some of the gamification tactics, which, if you haven't used duolingo, they are all over the place. There are points, there are gems, there are crowns, there are streaks, there's leaderboards, there's everything. And so maybe we start with streaks, which is that are kind of the cornerstone to some of the ongoing engagement. Notice that there are daily streaks. So talk through the decision to have daily streaks versus, you know, you could imagine, weekly or otherwise.

The fundamental reason we chose daily and streakk has been a part of duolingo almost from day one. And the reason we chose to start with daily is pretty much related to language learning. So it's pretty easy to forget concepts if you wait a week to do something. And it's obviously so much faster if you do a daily practice. You learn new content and then you repeat them with a quick cycle. So daily just met how language learning works, which if you can do daily practice, it works better. And another one is the strength of the habit. So if we can get people to build daily habits over time, the compounding effects of how much you can learn is so much higher than if we were to create a weekly habit. Two reasons. One is it really fits the language learning. It increases efficiency if you can do something on a daily basis. Two daily habits take you further because the compounding of how much you learn is so much better than if you were to try to do weekly or monthly or something longer than a daily cadence.

What about.

Because you could also imagine a world and, you know, high school classes do this where they get you in a seat for 60 minutes and pound English, Spanish any course into you. So I think there is this idea that maybe daily and shorter spurts is good, but it's definitely different than our current learning philosophy. So do you guys test these things?

We test a lot of different hypotheses. I think one of the challenges we have is we're a mobile company and your attention is very much fought over when you're doing anything on a mobile phone. So it's actually very, very difficult to get anybody to do something for 30, 60 minutes unless they're doing something very fun, like watching a video that takes less brain power to do. But as you're doing dueling you're constantly one of the pieces we have to make it fun is like we're never letting you read stuff without doing anything. You're always answering questions to learn. That's a pretty active thing to do. And on mobile, your attention is also fought over. So it's pretty difficult for us to get our users to do things for like hour, two hours at a time. That's why we try to make everything bite sized, so you don't feel the pressure of like in a class, if you don't complete the 60 minutes, you don't have much of an option. Right? You go to class, you do the 60 minutes. With us, you can do five minutes, but if you're compelled to, you can keep doing those five minute chunks to add up to 60 minutes. But at least in a very quick span of time, like five to 10 minutes, you can kind of extend your streak and get the benefit, at least if you don't have the 60 minutes to invest. So I think we certainly try very different lesson lengths, for example, and it makes a huge difference. Like, there's a sweet spot that we found with our current lesson length that, like, even if you make it one more exercise, you start seeing drop offs because not everybody can sit through that long. If you make it shorter, there's potential to increase our retention rate. But then you got to show people a lot of lessons to make them cover a lot of content. So we test a lot of these dynamics.

What's the sweet spot then? Do you have like three lessons, six minutes?

The one we've found, the sweet spot that we're at now is average time is around like 14 minutes. And we run a lot of experiments to increase it. But some people do it in multiple lessons, some people do it in one lesson, take their time with it, it changes depending on how much difficult content you're doing, et cetera. But yeah, there certainly is a sweet spot that a lot of people average around.

Okay, I promise we're going to move on, but I really have to get back to streaks for a minute. Companies like Duolingo or Peloton or dieting apps have figured out that streaks can become the main motivation for staying engaged, maybe even more than the activity itself. There's even one study that looked specifically at Duolingo streaks and found they do measurably boost motivation and attention. This works for most users, but especially for advanced ones. But there is a catch. For most of us, breaking a streak is incredibly demotivating. Imagine you're at 500 consecutive days of anything, then you miss and you're back to zero. If you look at the literature, you see that even though streaks themselves can be motivating, breaking a streak actually makes people less likely to start the activity again. But Jem says Duolingo's found a way around that.

Do you see a drop in engagement when people fall off their streaks?

Yeah, the streak is a double edged sword. So the fact that falling off the horse after 500 days is not a good event also lets you keep your streak. But obviously life happens. So sometimes you do lose your streak. It's not super fun to restart because for another 500 days you won't be beating your last score. It takes quite a bit of time to get back up there. We do a few things actually. So we've actually gamified the streak process to make it less daunting or make it easier to keep your streak. One thing is you earn gems as you go through duolingo. That's one of our virtual currency. You can spend those gems to buy streakk insurance. We call it streak freeze. So if you plan ahead, you can buy streak freeze and then if you don't show up for a day, your streak is frozen, it doesn't break. You could buy two freezes. So for two days you're safe. Pretty much. So that's kind of a prevention method.

And some people would call this cheating, but you guys, it's gamification.

Exactly. So you'll see some of our hardcore users on Reddit saying, oh, you have a 500 day streak, but did you use a streak freeze? It's like, are you legit? Did you use any streak freezes? But I think our intention isn't to create either cheating or like more hardcore use. Our intention is to make it slightly easier for people to keep their streak because life happens. I mean, keeping up the thousand day streak, you might be sick one day. And we don't want to pay penalties for anybody that might have very valid reasons to not duolingo every day and have the flexibility. So that's why we have kind of an insurance in place. What we do to not make streakk devalue is kind of cap it at 2. So you can't buy more than 2 streak freezes. So you have to come back to the app after two days if you want to keep your streak. Pretty much. And then another one we do is actually part of our monetization efforts is if you lose your streak. So you, let's say your insurance couldn't save you and you lost your streak, you could buy plus to repair your streak. So plus has a one month, every month you get one kind of repair. That's what we call the items streak repair. And if you're a plus subscriber, that also creates another prevention method of keeping your streak.

Got it. This is nice failure mode for some folks. So do streaks help people who are just starting? It seems like they're highly effective. If you've kind of gotten over the hump of I'm going to do this thing and I will be there every day for five minutes, give me my checkbox. What about in the very beginning? How do streaks work with like the person who's trying it out and seeing if this is their jam?

It works very strong for both users. So if you sign up just new today, we give you a lot of guidance on what to do. Our language tree is designed in a way where you're not really lost. You start from the beginning, you work your way downwards. You kind of have direction. However, how frequent should you practice is pretty much an answered question through the streak. So it gives new users pretty much a structure and a guidance to follow. They do some stuff on their first day and the next day they have a marker to shoot for which is, you know, they start a one day streak. They can go to two, they can go to three. Obviously the mechanic is stronger when you get to higher levels because if you hit let's say 30 days is kind of a place where we really start seeing very high retention is one, you've built a habit, you've done it for 30 days. And you might know this better, but a lot of research says if you pass like 21 days, you're kind of like on your way to forming a daily habit.

Can I tell you a hack? Yes, the 21 days is actually a.

Is that false?

It is false. It's basically. I don't know what year it was, but the lore is that some Botox doctor basically said it takes women 21 days to recover from some facial surgery. And we'll correct that in the show notes if I'm incorrect in the exact. But it's a really. It has been blown up to me and kind of everything takes us 21 days.

Okay, I won't keep quoting the 21 then.

Yeah, the reality is it takes, you know, anywhere between exercise could be four times a week for six weeks to one latest study said six months. Okay, so if you guys are at 30 days, this is rockstar and we should, we should do a study on this. Maybe we actually asked. So are people coming Back at the same time every day. Are they coming back in, you know, morning and night? What's their actual time routine?

Everybody actually has a pretty different routine depending on what their lifestyle is. But one thing we do structure is practice reminders. So that's a big portion of creating stickiness, which is push notifications. But obviously we try to be very strategic. If you use Duolingo today, anything we send you for the next seven days is basically what we call a practice reminder. It says, come back to Duolingo, practice Spanish. We schedule those pretty strategically to be exactly 23 and a half hours later after you've done a lesson today. And we've experimented many different time lengths before and that was always the winner. And it's the intuitive reason that, for example, your schedule allows you to mainly practice in the morning and then your practice takes somewhere from 10 to 30 minutes. If we schedule the notification in 23 and a half hours later, it's probably going to trigger at the right spot where you have free time to practice. So we kind of try to get you to develop a habit of doing it at the same time. But if we look at across our user base, people do it in various times. Some people prefer the morning, some people prefer twice a day, morning and evening, some people prefer evening, some people during the day. But obviously the peaks are non work times. So morning is a peak, evening is a peak, and the day is kind of less popular.

So there's this clever question you guys asked at the very beginning of when you're signing up during onboarding, and it basically says, how much do you want to practice? And it's something like 5, 10, I think 15, 20. So tell us about that page. Have you guys experimented with it? Does it actually correlate to what people do?

I think a lot of the questions you'll ask me about have you experimented with this is the answer is we probably ran more than a hundred experiments on most of the onboarding pages. That page we call the daily goal page. So when you're signing up, even before you getting into the weeds of Duolingo, we intentionally try to get you to set a goal and we label those goals, such as five minutes a day is, I think, casual. Ten minutes a day is serious. So we kind of add some labels so you can self identify on how ambitious you are and how much time you want to spend. And that marks what we call a daily goal every time you come back. So a daily goal is basically this mechanic where if you reach that goal every day, you unlock a reward so let's say you picked your goal to be 10. Generally you can hit 10 with two lessons. And if you do two lessons, you unlock a reward chest. And the reward chest has a random reward in it. That's also part of our, one of our gamification mechanics. You don't know exactly what you're going to get, but you get a random amount of virtual currency that you can use to spend to do other things. That's pretty much the goal of Daily Goal. And it creates this mechanic where you're not only to show up and do one lesson and leave. If you picked higher daily goals the next day you show up, you at least have a marker to shoot towards. So you do more than a single lesson and you do maybe two or three because you try to hit that goal and get your rewards.

So do you think people sandbag this at all? Kind of give you lower goals so that they, they hit them?

I think it's certainly up to you to sandbag if you don't want to spend too much time and get the reward easily. But we have a lot of users that are, I actually want to spend 30 minutes and I want to set an ambitious goal and I only want to receive my reward then. Mainly because I think Duolingo has two different types of motivation. One of them intrinsic, it's people want to learn French. The other one is this extrinsic, which we create through gamification. And if you're very intrinsically motivated, you're probably not going to sandbag. But if you just are there to check out things, you might sandbag, which both are totally fair use cases.

This is actually a really good point. A Meta analysis of 128 experiments found that tangible extrinsic rewards tend to undermine intrinsic motivation. So after people get an external reward, like money or prizes for doing an interesting activity, they tend to find the activity less interesting. This makes them less likely to do it spontaneously than they were before they were rewarded. There was a clever study that gave some kids a prize for playing with markers, while another group played with markers without getting a prize. Later, the kids who had been rewarded were less likely to want to play with markers again without another prize attached. The kids who didn't get the prize, they continued playing. And it's not only tangible rewards that undermine intrinsic motivation. Basically anything that undermines autonomy, like threats, deadlines, directives, pressured evaluation or imposed goals all can make us less likely to do something. By contrast, choice, acknowledgement of feelings and opportunities for self direction can enhance motivation. But I should Add that not all rewards crowd out intrinsic motivation. In a situation where interest in a task is low, rewards can actually increase interest. And this is key. Sometimes these studies that throw extrinsic rewards under the bus seem a bit off to me. If someone doesn't have that intrinsic motivation, what now? Getting intrinsic motivation is incredibly difficult and frankly, I don't know many robust studies that show how to do it.

We see certain types of gaming the system behavior because and we mainly see this on leaderboards because the extrinsic motivation from the leaderboard is winning the leaderboard. And for some users that overtakes their drive to do things that are more learning focused and they do maximize for just getting more points in a given a span of time. For example, you might go to your easiest skill and do that skill over and over again. And even though that's not teaching you new French that much, it might get you a lot of points fast because you kind of know all the answers but you're not learning new content. There is some of that behavior. I would say it's not necessarily a major problem because I think the intrinsic motivator to learn French is still very strong. That's one, two we control for a lot of what we call XP farming, which is so if you're doing the same thing over and over again, there are diminishing returns. So in a span of a day you can't just keep spamming the system I guess. So there are checks and balances in place so it's more of a fair competition.

Nice. I'd like to see that push notification that says stop, stop trying to cheat. So since you do so many A B tests, do you have a memory of kind of one that you thought would work but didn't? Or any kind of surprises in the A B testing game?

Our Duolingo plus subscription page we also experiment a lot on how we talk about plus what we present. One of the changes we did was we have a free trial to enter Duolingo plus and we ran an experiment to change the button copy from I think what it said was start my free trial. When you're starting your free trial to say start my free week you had a seven day trial and that was we expected kind of not much out of it. It's the same message but turns out start my free week. Our users responded a lot better too because it's probably solidifies the amount of free things you're getting with the trial. Small test, but love was a pretty interesting result on the amount of uptake we saw from just a simple test.

So how did you land on week? So you can imagine, I don't know if in the past have you done 30 days, two weeks, has week always been kind of the winner for you?

We tested a week and two weeks. We've actually even tested shorter periods. So one week or two week performed somewhat similarly. We currently have a two week trial. We tested our way into it pretty much. But the pattern we see is Duolingo users pretty much have one pattern during weekdays and then potentially a different pattern in the weekend. People who have a streak maybe do the same thing across. But some of our users behave pretty differently weekend versus weekday. So at least a seven day period gives them a full kind of life cycle on using their duolingo the way they're used to. And then in a span of seven days they get kind of a full experience at least. Why we thought seven day performed better than anything shorter. Two weeks is what we have right now actually is slightly better than one week. Mainly we think because it's a better poll, it's a better deal if you're buying a monthly subscription, you have more time to decide if you want to pay or not. So right now we landed on two week. But one week or two week kind of fit the sweet spot, I guess for what our users were looking for.

Are people understanding that they want plus right away or does it take them a while to click the trial button?

It's actually all over the place. And meaning some of our users, they sign up on day one. They are all right, I understand your free product has ads. I don't want to see ads. I will sign up on minute one. Even like they sign up duolingo, they do their first lesson and then they buy plus right after. But we also have users that have been using the free product for five years, six years, for a very long time and on one day they decide to upgrade. And their reasons for upgrading differ quite a bit. I think our new users just want to see an upgraded experience that they can get with. Plus they don't want to experience how free might be and upgrade on day one. I think our users that have been around for a while probably do it for different reasons. Like they want really want to experience the feature that they've been exposed to but haven't fully seen. Or we also have quite a bit of users that support our mission and they are. I've gotten so much use out of duolingo. I understand duolingo stands for access to education. I will upgrade, get the benefits but also I want to support the mission of Duolingo. We have quite a bit of older users even subscribing for that reason as well. So yeah, it's the time on when subscribes and why is pretty mixed.

So I'm interested for the Duolingo plus, how did you experiment into the features that are in the offering? Because it does seem, especially since going public, the monetization has worked, or at least something about it has worked, that people like or what feature is kind of the killer feature in there.

So if you look at our past material, Duolingo's pitch around monetization was no ads, no in app purchases and no subscriptions, which is exactly what I do today. And in 2016, we started with consumer monetization, which our first experiment was ads. So we introduced ads. It was our first monetization experiment. It was one way to monetize without asking anybody to pay for anything. But as we introduced ads, we got a lot of user demand for, well, I have money to pay. Can you sell me no ads. So we were okay, well, we can try that. That sounds pretty straightforward. So that was the first monetization feature. Another one we put into the first version of plus was offline access. So if you're traveling on a plane, you want to do your Duolingo, you could download the whole course. And that was the first version of plus. So no ads offline. And we also had a feel good feature, which was the third one, Support our Mission, which wasn't really a feature. So that was the first version of Plus. And then we experimented our way into. Now we have seven or eight features at this point in the whole subscription. We experimented our way by looking at where user demand is. Are we okay charging for it? So obviously we have a pretty strong line on where we draw and if it's good for everybody's learning, we're not going to monetize it, even though there might be a lot of demand for payment. And we added a lot of features over time. One of the ones that is most popular now is what we call unlimited health. So we've introduced a health mechanic to Duolingo, which is, as you're doing new lessons, if you make a lot of mistakes, you run out of lives, just like a game. Or you can buy plus to have unlimited lives. And that is currently our most popular feature, together with no ads, which also still has a lot of demand.

Wait, can you break that down?

You have five lives. Every mistake you make costs you a life. If you run out of lives, we stop you from doing new content. You can always practice what you've learned. So you're not stopped from using Duolingo at all, but accessing complete new words that you haven't learned new lessons. We kind of stop you if you're out of lives. You can refill your lives by doing practice. And the idea behind all of this was if you're making a lot of mistakes, you're probably entering territory that you don't know very well. If we can get you to practice, it's a better way to enter new content because you can remind yourself of what you've forgotten. So that was the idea behind lives. It's obviously also a pretty common mechanic in games, but if you have plus, you don't have to go through this mechanic pretty much. You have unlimited lives. You can go at your own pace and choose to do practice if you want, or expose yourself to new content.

This is really interesting. So there's a theory of learning that essentially says you make a bunch of mistakes, you realize the right answer, and that helps us retain the information better than if we'd gotten the question right in the first place. Having five lives is a nice way to reinforce the importance of recall and testing. It puts some weight in behind each question.

I think the one thing this mechanic did is you pay a lot more attention when you're answering. So the meaning right now, if you open up duolingo and without the help restriction, you can just answer whatever. And on a mobile app, it's pretty easy to not pay attention and answer whatever. I think if your mistakes are finite, or at least there's some repercussion to making a ton of mistakes, we did see a lot of users, their correctness rate go up in their exercises. So the accuracy went up when we introduced it. From a learning standpoint, obviously it's very normal to make mistakes as you're learning new content. I think we added this mechanic generally as kind of an incentive to also do practice. Doing new content is more satisfying because you're conquering new territory in a way in your learning journey. But practice, you do also need to do pretty constant practice to get better at language. So this was more of an incentive creation for practice, obviously also with kind of a plus feature attached to it as well. So to answer your question, your question was, which one is better? Obviously making more mistakes is expected, but I think also answering the questions carefully is what we achieved with the health mechanic, which was a win on its own.

Do you have this concept of active and passive learning, like you want people to kind of feel like they're struggling a little bit, even though it may be painful.

Maybe we don't necessarily use those terms like you did, but we try to shoot for an accuracy rate, which we've found if you can answer 85 to 90% of the questions that are given to you correctly, that's when all of our metrics maximize in general, like retention, time spent, etc. Meaning if you answered everything correctly, your content is too easy for you, so you're not going to feel very good about the lessons you did. You might not be learning as much, so you need to have some error rate. If we lower it to like 70 60%, it becomes frustrating. So you don't get a lot of joy off of doing duolingo lessons. So we found the sweet spot for engagement is around 85 to 90% accuracy or correctness rate. So that's what we try to shoot for.

So in learning science, what Jem is talking about is called active and passive learning. Active learning feels difficult because you're actually making mistakes. And even though I said earlier you retain more when you do make mistakes, it can feel like you're not learning. Passive learning is more like like listening to a lecture. You're taking a class, you're enjoying yourself. You think the lessons are fun, but because you're not engaging with the content that much, you're likely to learn less. Duolingo plus users are paying their way out of active learning and buying more passive learning. And I get it. It makes sense that people want this. It's not fun to be wrong. One classic example concerns an incident at Amazon involving a revision of the mobile game Air Patriot. After Amazon launched an updated version of the game, the seven day user retention rate immediately fell by 70% and revenue fell by 30%. What happened was the game's developers had accidentally made the game more difficult for users. They fixed the problem, but then wondered if making the game even easier could produce larger increases in retention and revenue. So they ran a test with four new levels of difficulty in addition to a control and learned that the easiest version did the best. Amazon launched a newer and easier version of the game and sure enough, both user retention rate and revenue jumped by 20%.

Hooray.

So people like and engage more with things that are fun and easy. Maybe this isn't surprising to anyone, but it is concerning if you're duolingo because we need errors to learn. Learning when it works feels hard, and if this is the case, one can imagine that it does interfere with a frictionless gamified habit forming experience.

When it comes to active versus passive, we actually use that pretty differently. Where most of Duolingo is active learning in the sense that you're always engaging and interacting with content. Whereas we've also in the past few years entered into what we call passive learning, which is you can just listen to content and learn without having to answer questions. That's why we created a podcast. For example.

I give some lectures and people love to hear behavioral science principles, but if I don't make them answer questions about that, I know that they leave the lecture and will forget. But they love hearing it. Maybe actually we dive a little bit more into kind of the theory of learning. Like we've talked a little bit about errors. Are there other tactics that just really help people with either retention or fluency or at least on the journey to fluency as you mentioned?

I think there's probably a long list, but I'll try to remember some of the main ones. One of them is making it bite size. I think we talked about this. The other one is not giving a lot of instructions to read. If you look at Duolingo's exercises or any page that you'll see on Duolingo, it will have the minimal amount of text, minimal instructions and intuitive simple user interface. But it's a big contrast to how language is learned in general, which is through a textbook where which is a lot of text instruction you go through. You first read a conjugation table and then you try to exercise. What we've found works better is letting you jump into the exercise. You've never seen a conjugation table. You're giving hints and then you're answering questions and then we unlock new concepts for you rather than giving you upfront instruction to read, which a lot of people generally we've tried those things in the past. It's retention drops off pretty quickly. It's not as fun to read a conjugation table and then do exercises. Whereas if we can get you to the interactive answering questions mode, it's more fun. And then we can expose new concepts over time by doing so is what we found works really well. The other one is so Duolingo has this slightly complicated mechanic now which is if you're doing a skill and skill is a collection of concepts that we are teaching you. If you're really interested in that concept, you can keep leveling up a skill which will give you the same set of words but harder exercises on those words, on that vocabulary. Or if you just want the basics and Just want to learn more vocabulary, you can move on to the next skill and just learn the basics of those words and not go too deeper. One example I'll use, it's on a level one of a skill, you'll do simple translation exercises. On a level five of a skill, you'll be asked to write a Spanish sentence if you're learning Spanish using those words. So it's obviously a lot more difficult to do a level five than a level one. And not everyone is going to self select into doing the hard ones. Some people are just for brain using duolingo for brain training or other reasons. So we let users self select into how much difficulty they want to enter into. And that's worked really well for our metrics in general.

Interesting. Are you nudging them at all into their so in behavioral science we tend to be more paternalistic where we try to nudge people into things that they think that they want, but maybe in the moment are distracted or have other things coming up. So how do you think about kind of nudging people into harder levels?

We actively try not necessarily nudging one direction or the other because we found the optimal way to learn is actually do both. So meaning we want you to level up the existing content you've learned so you can do the harder exercises for the same set of vocabulary, but we also want you to kind of move down the tree. That's how the kind of our language tree works down the tree and get exposed to new vocabulary. We published a blog post that we found is kind of. I mean we found the most efficient way to learn on Duolingo is doing what we call the hover method, which is you hover over the beginning of your tree. That's the content you've already learned. You practice them, you level them up, but you also go down the tree which is learning new vocabulary. Doing the combo is the optimal one. To be honest, we haven't fully figured out how to kind of lead a user to use the hover method. We let users self select into how they would like to navigate the tree, but we actually have a full team now working on how to create an efficient path for duolingo. So it's not fully self selection and can we nudge our users to the more optimal path? It's a hard problem, so I'm not going to claim we solved it, but we are trying to be fair.

Learning a language is a hard problem. And on mobile, as you mentioned, we have all these distractions coming at us and so it's possible level 5 is also competing with my WhatsApp notification. Okay, what is changing tactics a little bit? What is a contrarian view of behavior change that you've developed either in your past, but maybe add Duolingo. You know, everybody thinks this works, but I think this works.

This is kind of a recently formed one. I think generally when building product, the most intuitive process for building good user flows is make the flow as short as possible. So if you'd like to be able to purchase flow, put everything into a single page. Because if you have multiple steps in a flow, people will drop off and not wait for the last step. For example, selling plus has never worked well. If we put things into a single step, it goes counterintuitive to general product management or design principles, where if you have multiple steps, we actually have more time to teach people about what plus is, how their free trial works. And that performs so much better than trying to put all the information into a single page, like how much it costs, how the free trial is. It's just too much information overload in a shorter flow. So building longer flows that kind of explain what we're trying to sell with Duolingo plus has worked so much better. So I would say that's one thing that's contrary to what anybody would say about product design.

Yeah, I love it. Have you tried different ways to explain things? A personal pet peeve is mine is where you have just carousels, where people just swipe through and we know that they're not really reading. And. But some designers thought long and hard about what goes on that carousel. And so while we can make them a little bit longer, these carousels make me worried that people just aren't paying attention. How do you actually make sure that they're kind of understanding as they go through the flow?

Oh, carousels don't work. Is the other contrarian potential contrarian, meaning?

You heard it here first.

Yes. To use a stat, we've tried many different design patterns, either onboarding flows or plus onboarding flows. And we did have a carousel that kind of talked through all of our features. 80% of our users never even went to the second step of a carousel. Carousel generally are not super intuitive, and the yield you get off of building a carousel is from the first slide and because people don't see anything else. So whenever we're doing any designs for new features, my approach is don't do carousels.

Yeah.

So maybe if you're listening, don't do a carousel. If you have a carousel, kill the carousel and just focus on something else.

Yes. And even a visually not good looking checklist might perform better than a really good looking carousel is what we've found.

I wanted to try an onboarding question that looks something that like highlights the benefits. So it's like how much better of a person will you be after you've learned the language? You know, how proud of yourself will you be kind of more highlighting the idea that Duolingo will provide me all these benefits in life and instead of just telling people, can you ask me, you know, 1 through 7 how happy I'll be with myself and maybe people will kind of actively engage in a little bit more.

Every time we've collected data or done research, there is no clear 2, 3 use cases that pops out. Everyone is learning for different reasons. And it might be like the school use case you mentioned. They might be taking a French class and in class they might be using Duolingo. That's very specific. Or you might have French grandparents that you want to practice conversation with or you might be going to France. The use cases are very broad. So it is. We try not to boil it down to, oh, there's these two, three big use cases. Let's cater our product to them mainly because that would leave a lot of our user base outside. And there's also the other one which is a very different use case. People learning English across the world and that's a lot more of a serious use case for most people. They might be trying to get into an English teaching university, trying to get a better job. That's a complete different use case in general than what we see in English speaking countries, which is less about career or school. There are all these different use cases, but internally as we build product, we try to build for all of them rather than try to pick one and optimize our either messaging or product features on any of them because they're just very different. There's a huge long tail of various reasons people learn for. So yeah, that's why our product choices are more broad than narrowing down on any specific Persona or use case. We've actually thought and brainstormed a lot about can we get you to set higher goals, midterm goals or long term goals as you're signing up to Duolingo to be I want to learn French in six months. We've actually never built a feature like this, but it's been on the table for a while so there's a chance we'll try something that that is similar. It's certainly the highest what we do see is the highest point of motivation anybody has is day one because they haven't fully realized how much time it takes to learn a language. But they're very motivated to make progress. And at first, as a beginner, you are making very good progress. You're learning a lot of words. So the rate of improvement is really high. I think the toughest part is you're midway. So you're midway. You still realize to get to fluency, you maybe need six more months or however long it might be. And one thing we're trying, which isn't exactly what you propose, one thing we're trying is creating these monthly goals, which is every month it resets and says, hey, do 2000 XP, which is quite a bit of work, and win your monthly goal. So at least it creates this midterm incentive to solve some of the midterm motivation problem.

I like that. We've also seen that the highest part of motivation is at the very beginning. And this is kind of when you can capture people to lock them into future goals. And I think the goal of any kind of engagement and retention strategy is to make sure you capture them long enough so the mid midway point doesn't fall flat. But intervening. I like these monthly goals. Have you tried calendar things where you're putting stuff on people's calendars or helping them kind of block time?

We have not. What we do is show you a street calendar. That's one of the things that is tied to the streak. So we show you kind of a normal monthly calendar and we mark the days you use the street freeze. So if you're being honest with yourself, you can see kind of how much freezes, insurances you've used and you can get the, I guess the visual gratification of your streak kind of filling up a calendar with we used an orange fiery color. So the calendar fills and there's a visual satisfaction that comes off looking at your street calendar.

How do you bring somebody back after they've given up? So maybe they've hit that midway point and they're just not coming back as much as they used to. Maybe they picked up another hobby. How are you reeling them back into the learning journey?

We do a few things, actually. I'll mention a couple. One is if you drop off today for the next seven days, we're going to try to bring you back with what we call practice reminders for seven days. And a lot of those push notifications will say things like, don't lose your habit. Come back to practice French. Pretty intuitive. Come back, don't lose your habits, then we stop. So obviously we don't want to be spammy and send notifications forever. If you've decided not to use Duolingo, we do one last shot which is on day 30 of your dormancy. So you've stopped using Duolingo. You haven't used it for 30 days. On day 30 we do something that we internally call the passive aggressive reminder, which is pretty self explanatory. And so we have our mascot duo which we haven't talked much about, but he's kind of your language coach. He is a green owl and we use his voice to send this notification that basically sounds like a pissed off spouse in a way and says it seems like my notifications aren't working, I'm going to stop sending them for now. So it's kind of a. It is a passive aggressive tone and it actually works pretty well in getting users to notice that push notification if they kind of gone blind to them and react pretty positively to being I actually didn't want to drop off and thank you for taking this weird tone with me. Let me go try it out. So we do push notifications and emails pretty much to kind of get people back. And after day 30 we stop. We don't try to. Obviously we're not trying to be spammy, but we still do see a lot of people after day 30, whether through the push notification or on their own come back and resurrect. Actually more than 15% of our users on any given day is what we call resurrected users, meaning they've been dormant for more than 30 days.

You're lucky in some ways because daily actives probably correlate to learning and so you have this nice overlap. But is there another metric that you use for learning? So if somebody actually kind of can or tan can speak can ask for coffee at the coffee shop.

Measuring learning on a single graph is pretty tricky. Like you identified, there is no clean graph where we can point to and say, oh, learning is going up. Because there are many different ways to measure it. But we do use a few proxies. One of them is like you said, daily activity. The other one is time spent. Learning is one thing we measure. So anything that you spend time on on Duolingo that has a learning element. So if you're staring at your tree, that doesn't count. But if you're spending time in a lesson that counts, we measure time spent learning as another one. And a third one is a bit more of the complicated measurement that we do. So throughout your Duolingo experience. There are quizzes that you take that are blended as an exercise, but we use those quiz responses when we make, for example, the content changes or design changes to measure in aggregate how much the learning output is changing. It's not perfect because test design really impacts what score you get. So. And test design could many be done in many different ways. But we try to use kind of our learning quiz results to also infer if we're increasing learning. Another inference we do is difficulty. So if our users are experiencing more and more challenging challenge types, so we have different challenge types in every skill, we kind of infer that, all right, we increased exposure to more difficult challenge types, which is generally better for learning as well. So we look at, I think, a combo of all of these to decide, all right, have we taught better? And if, then if we run an experiment and all these metrics go up, we can conclude generally that it's teaching better.

Okay, last question. So the first time that an actual film was filmed, they put a camera out, and in hindsight, you look at the movie and you're like, why didn't they move the camera around to get different angles? It's just a stationary camera. And the kind of hindsight lesson is like, they just didn't think about it, like, it's a new thing. And like, you just didn't think about moving the camera around to get different angles. That's because they were early in the journey of a film. Are we early in the journey of, you know, language learning on mobile? And what are kind of our blind spots where we're doing something now, but in 10 years, we'll look back and be like, what? Why weren't we doing X and Y?

Oh, I love how you phrase this question. And I'll try to be concise with my answer here, but fast forward 10 years, I think one thing that will be pretty obvious is the answer to, oh, how did you learn French? And the norm will be, oh, I learned it on my mobile app, which the analogy I use is actually dating. So if you ask, it's a very generational divide. But if you ask a younger generation and say, oh, how did you find your partner? Majority of younger generations answer, I think anyone below 30 is going to answer, oh, on a dating app. And 10 years ago that was not the case. So I think over time, these norms change. And I think fast forward 10 years, the norm will be, you learn a language on a mobile app. I think we're headed in that direction mainly because mobile apps can teach you a Language very well, but I don't think the perception around the world is there yet. I think a lot of the perception in the world is still, oh, the best way to learn a language is go to a class or hire a tutor. There's a big perception shift that will happen over time and it will take time to do. The other one is fast forward 10 years, apps like, I mean, hopefully Duolingo will get there. We will teach the fluency. I think that's. It takes time. It's very difficult to get there today. If you know nothing and pick up Duolingo, you won't be fully fluent. But we are putting majority of our product teams to work on pretty much how do we teach better and compounding effects. For, let's say 10 years, you will pick up Duolingo from zero and by the time you kind of put it down and finish all the content, hopefully you'll be fluent. So obviously it's not a fully linear path to get there, but in the grand 10 year timescale, I think that will happen. Mobile apps will be the norm rather than the exception. I think if, if somebody says, oh, I learned French on a mobile app, you were like, cool, how did you do that? I think in 10 years the norm will be, all right, which app did you learn on? Or hopefully what is your Duolingo score? Duolingo will be the norm of that as well. But, you know, that's kind of how we view the future, I guess here on how things might change, I was.

Hoping you would say something like VR. Where we are now in, you're putting out a VR headset and you're going to the country to learn the language. But maybe that's in 20 years.

I'm very bullish on VR. Yeah, it might happen. But yeah, the headsets now seem kind of janky, so I think it might take a little more time.

We just got the mobile thing going on and that's good to go.

Great.

Thanks for coming on the show. This was really lovely, super fun conversation. I learned a lot. Likewise, I'll get back on the app and keep my streak going.

Please do.

That was Jem Cansu of Duolingo.

I gotta say, I feel for this guy.

He has a tough job because learning a second language is tricky. So we learn languages in one of two ways, either explicitly, like those verb conjugation tables we all remember from high school, or implicitly, which means you're exposed to a language and you pick it up as you go. There's one popular meta analysis out there that shows how explicit second language instruction tends to be more effective than implicit instruction, although both methods are useful. Like most language learning apps, Duolingo leans more on the implicit side, but it's probably more useful to learn a rule for conjugating a verb into past, present, future than to extract the rule from a bunch of sentences. And then there's the whole gamification thing. I said earlier that the behavioral science world doesn't exactly love that word, and Duolingo is really leaning into gamification. But in their case, is that necessarily a bad thing? Obviously the research is out there. Offering points and prizes can reduce our intrinsic motivation to do something, but I think we have to wonder, how else would people learn? More specifically, how else would we learn a language as an adult, which is incredibly difficult and time consuming when we're all super busy? What is the recipe if you don't have intrinsic motivation, but you still want to learn a language? I'd propose that for people who don't have that magic motivation, points and badges can be quite motivating. Of course, they can also motivate people to cheat their way to completing an exercise just for whatever fun currency they might win. And then how much are they really getting out of the app as they evolve? That's what Duolingo is going to have to answer. The Science of Change podcast is presented by act on the Moments to Lead to a Sale. Visit them at Setsale co or on LinkedIn to learn more about the world's number one revenue execution platform. This podcast is produced by me, Kristen Berman and StudioPod Media. Our executive producer is Rachel Roberts, and all episodes are written by Jack Ewer.