How Does Airbnb Disrupt with Design?

Behavior change is hard. Actively changing another person's behavior is even harder. But there are plenty of companies out there that have been able to crack that code. They understand that people don't always or.

Even rarely act rationally.

I'm Kristen Berman, co founder of Irrational Labs. Over the years, we've helped countless companies understand and leverage behavioral science to increase the health, wealth, and happiness of their users. The idea behind the Science of Change was to create create a podcast that demystifies the psychology of decision making. So here's how it works.

Each week, I get to chat with.

Visionary product and organizational leaders to uncover their insights about user behavior and how they've managed to change it. I match this up with some of the latest research in the field to get to the bottom of our core psychologies. So let's dive in and unlock the secrets behind the science of change. The Science of Change podcast is presented by Set Sail, world's number one revenue execution platform. Today, we will talk with someone who has built not one, but many products that people love and ones we all know.

Joanna Zhang, better known as Jay Z.

Is currently the VP of Product and Webflow and LED product teams at Airbnb, WeWork, and Dropbox. She was also a lecturer at Stanford University. But what I really wanted to go deep on with Jay Z was, was Airbnb because it's a company we all know, we've all used, and it's just steeped in classic behavioral science. We touched on everything from what they do to drive more reviews, how they encourage behavior change, and how they manage to get hosts through that onboarding process, which is no easy task. JZ understands human behavior, what works, what doesn't, and brings a unique perspective that she's picked up through a lot of at bats and a lot of trial and error, although it doesn't seem like there have been many errors. So let's dive in.

Okay. Jz, who are you? Do you want to explain your past and your present roles?

Sure. I'm currently the VP of Product at a company called webflow. We basically empower anyone to build for the web. And before that, I worked in product at companies like Airbnb, Dropbox, WeWork, and I started my career in product in mobile gaming.

Wonderful. So we're going to dive into some of your Airbnb experience and we'll start off with basically some of the bread and butter of what you learned there. So was there an experiment that had surprising results?

I would say that so many experiments actually have surprising results in that you don't. You really don't know what guests and hosts are going to do. But I would say that one experiment on the host side that was quite surprising is when you think about onboarding flows, typically, you know, the shorter the onboarding experience is, the faster people are able to make it through and, you know, generally the more successful they are. But in reality, for hosts, putting in more information so, for example, actually uploading more photos instead of less photos and adding captions to those photos and adding, you know, a longer description, all of those things actually make them much more successful because it makes their listing more bookable, which ultimately is a success metric for a host, publishing that listing, but then getting it booked by a guest.

And so some teams have the trade off between, if you ask somebody something upfront, they'll drop off. And so what you do is then you get them in the product and then you bug them, you nag them later to go add pictures.

Yeah, that's right.

Or captions or things. So what do you think about the trade off between forcing people to go through a long onboarding with high drop off or nagging them later to do the things you want them to do?

Yeah, so I think for products, you know, again, I mentioned, I worked at Dropbox and some other places for products like that, where the thing that you're supposed to do is rather simple. You are essentially uploading your photos or your files, you want to make that onboarding as frictionless as possible. So getting from point A to point B as quickly as possible, and then you can nag them later on to do other things and configure settings. But for something like hosting on Airbnb, it is a huge mental hurdle to be able to let people into your home. So. So, you know, actually what you want are people who are willing to take that leap of faith, who are willing to let a stranger in, and who are willing to take the time to really understand what Airbnb is about. And so in that case, for something like hosting, it's actually better to really make sure that that host is mentally prepared for what hosting actually means.

So let's double click on the onboarding. How do you tactically get people through long flows? What are the things that you can do to motivate them to keep going when they may have other things to do in life?

Yeah, I can share a couple of tips that I think would be applicable to any company. And then I think there are some things that are very specific to Airbnb, especially for a hosting onboarding flow. So for any company having A progress tracker that is really helpful. So knowing, hey, this is how far I am in my journey, and really showing that there's a North Star, a finish line, really, and how you're progressing against that, that is something that works well. I think the thing to be wary of is if that progress tracker is so far away, if you're, you know, 1% of the way done, I think then you have to think about, well, can we break that down into smaller milestones and get people to those milestones faster?

So we've talked about this in other episodes. People are really motivated by making progress, especially when they're close to the finish line. There's a study that analyzed Olympic swimming competitions that involve multiple laps, and the researchers found that swimmers actually swim faster towards the visual finish line than they do swimming away from it. In this sense, letting users see that they are close to the finish line gives them an extra push to cross it. But like Jay Z says, showing progress can also backfire. In one experiment, my team at Irrational Labs told people via email that a certain important process would take 10 minutes to complete and provided a link to get started. We hypothesized that more people would complete the flow. Why? By telling people and being transparent, we thought people would be able to set aside the appropriate amount of time. Also, 10 minutes didn't feel like a lot to us. Given the benefit of completing the process was massive. What happened? We were wrong. Fewer people completed the flow. When we exposed the amount of time it would take compared to when they saw no time expectations. Why is this? Isn't more information good? Sometimes. But if someone tells you it will take 10 minutes and you can't do it immediately, you may think to yourself, I will set aside time and do it tomorrow. Tomorrow comes and you forget the momentum was lost. So I was curious, how did Airbnb handle this?

Yeah, so for Airbnb, you know, when you're essentially onboarding as a host, there's so many parts to getting a listing ready, whether it's taking those photos, it's writing the listing description, and really just thinking about all the settings that you want to enable for your guests. So there's a lot there. And one of the things we learned that was more specific to Airbnb is uploading your photos is difficult. That is a big hurdle. And a lot of people don't have those photos ready or don't really know how to take them. And so we thought a lot about what platform would be best. Would it be desktop? Would it be mobile? And we really Discovered that on the mobile phone, that's where people are when they're thinking about their photos, accessing the photos, or even when they're taking their photos. And so really prompting our hosts to take photos on their mobile phone, actually even using clever things like GPS to say, hey, you're basically in your listing. Why don't you go ahead and take the photo now? Especially for a property manager who has multiple listings. And then later on, going back to your question, around when do you prompt in time versus later, later on, prompting them to add those captions before they're finally able to hit publish.

Yep. So this is kind of like just in time nudging, where somebody's at the home and you're like, look, now is the time to do the thing that we both agreed is important.

That's right. And of course, you know, every user is different. And so, you know, some people can do it all in one sitting, but really being able to segment those users to know that these are the ones who are really struggling with their photos, they only have one out of 10. And also knowing how many is the right number. Right. Like, we know that with 10 photos, you're more likely to get booked because we have enough photos for our guests to really see the layout of your home. So all of those things to help segment and then prompt the right users at the right time.

Yeah. I love giving people the anchor of how many is good. Sometimes it feels like people just ask you to do things without the goal of get to 10. And so when you look at how many photos people have, and maybe this is something we can do on Airbnb now is, do people have 10 because they're trying to hit the mark? Or how did you get them to get to 10?

Yeah, so it's a combination of the number, but also, as importantly, the quality. So you can take 10 blurry photos, and that's not gonna do you that much good. And so what we actually worked on was a machine learning algorithm that would look at the photos and say, this is a good photo or this is a bad photo. And that was trained off of a lot of data from our professional photographers. So a combination of number, you know, seven to 10, minimum. And more importantly than number, it's like, do you actually have a picture of the bedroom? Do you actually have a picture of the kitchen? Do you actually have a picture of all the spaces that a guest would want to access? And then on top of that, the quality of the photo.

Nice. This feels like a good feature for dating apps to do of some AI that tells you if it's a good picture or not.

Yes. Or like you're holding something weird here. Maybe you should consider. Consider not holding that axe.

Yeah. Low quality. Low quality. Great. Any other experiments that had surprising results or, you know, taught lessons to the Airbnb team?

One of the big things we did learn as a team is that it is very, very difficult to change behavior. You can change behavior just, you know, in terms of number of photos and quality of photos. That's a very simple thing. But telling someone, hey, your home isn't clean enough, please clean it better, or your check in experience is not the most amazing. Those are much harder things for someone to hear and then change their behavior. And so what we realize is there's always a limit to how much you can change someone. And instead, what's more important is to categorize properly and match better. So who are the guests who are very, very picky about cleanliness, and how do we match them with the places that are, you know, really stand out from a cleanliness rating perspective? And so a lot of it then comes to matching and the search algorithm, as opposed to specifically asking hosts or guests to change their behaviors.

So while changing behavior is hard, did you have any findings about how you'd get those small incremental changes? Like if you could tell somebody, look, your place isn't clean, or do you tell them your place is very clean, but it could get cleaner if any small ways to deliver bad news that were interesting.

Yes. So I think in general, what we did is we delivered overall news because we had their overall rating and their subcategory rating. So, for example, their rating for cleanliness, for communication, for check in. And so first of all, the first learning is just deliver the feedback. So we would always, you know, take whatever ratings there were as soon as there was a critical mass, call it three rating, so it was a little bit more anonymized, we would deliver that feedback. So the first learning is just deliver it. But then, you know, people respond very differently to that feedback. There are some people who crave the feedback and want to make the changes. And so there we learn to give them more just in time, tips, more transparency, more of like a host dashboard for all of the things that they could do differently, actually pairing feedback with recommendations of what you could do to make that particular piece of feedback go away. And then we also learned for the users who were not necessarily clicking into their dashboard, who were not necessarily doing anything to see their scores go up, that's where we could categorize them more on the back end and figure out how to propagate that through ranking to get them the right match.

Nice. So faster feedback loops. Maybe this is in generally a good Instead of doing quarterly performance reviews, doing faster feedback is probably good for most.

People to hear yes, always fast feedback and then yes based on the reaction, then decide how to guide Giving rapid.

And relevant feedback is a great way to cultivate self awareness and enable behavior change across different contexts. For example, in a medical intervention, researchers gave healthcare workers immediate feedback on handwashing compliance and this increased hand hygiene in the short and long term. In a large scale energy conservation field study, people who received real time feedback on their water usage used 22% less water than those who didn't receive feedback. And this is not only helpful in product design, it's helpful in performance reviews. Annual performance reviews are the opposite of fast feedback, so we shouldn't be surprised when they fail to help people actually improve their performance.

So basically the host is playing a support role. At some level, they are managing a guest's emotions, how they feel, and responding to them real time, sometimes, or after the fact. How did you either coach hosts to do this, or were there best practices on how to respond to angry or upset guests?

That is a great question. And it's very difficult because at the end of the day, Airbnb is a platform for hosts and guests to partner with each other and interact with each other. And so finding that line of that platform responsibility and what you can do and what you shouldn't do was very important, but also very hard. And so we did believe that, you know, it was our role to provide that feedback. It was our role to provide support for the guests if they needed it, in the form of customer support. So anytime that a trip goes south and the guest needs somebody to help that isn't the host, we're there. But we actually did try to maintain that platform neutrality when it came to really kind of facilitating that host and guest interaction.

Bad experiences are bad for business. When a consumer requests an Uber ride and arrives later than promised, let's say because the driver took a wrong turn, research found that the consumer spent 5 to 10% less on future rides than an equivalent rider. But bad experiences do happen. So for Airbnb, how should the host respond? It's tricky. In one world you could profusely apologize, but then you admit blame and you open yourself up to a negative review. In another world, you could defend a bit, not admit blame to protect your reputation, but risk annoying the guest what does the research say? Researchers did an experiment with Uber testing out different forms of an apology after late rides. Some were costly, giving angry riders a $5 coupon. And others were just written apologies, with some even expressing a commitment to be better. Maybe not surprisingly, the ones with a $5 coupon attached performed best. The idea is that an apology without any teeth may not mean much, but by inserting a cost to the transgression, you really signal that you care. The worst condition? Apologizing? Committing to be better, and then not. If the rider had another bad experience after the commitment, they spent less money on Uber in the future than if there had never been an apology at all. So does Airbnb nudge hosts to respond appropriately and effectively?

I mean, being a host is just very difficult. And I really have a lot of admiration for all the Airbnb hosts out there. Part of hosting part of hospitality is that the guest is not always right. Right. But at least deferring to the guests and really making sure that they have a great experience. You actually hosts encourage other hosts to do was to really, you know, think about that guest experience and really put themselves in the shoes of that guest. And so if that did mean apologizing quickly for things that didn't go exactly as planned, or if the guest wasn't feeling amazing in general, that does generate a better experience for the guests, which ultimately leads to a better rating for the host, which is again, ultimately what the host wants, because they want guests to have a great experience. That's why they are hosts. And when guests have great experience experiences and leave them great reviews, hosts actually get booked more in the future.

Makes sense. So maybe we jump into ratings, which it seems like the currency on Airbnb is, you know, that you will be rated as a guest, you will be rated as a host. And so we do things to influence ratings. Can you talk about some of the secret sauce behind how the ratings were developed and what they look like to improve upon kind of accuracy?

And yes, my team's worked really hard on the ratings because it is such an important currency. One of the things that we learned was it's really important to have it be double blind. And what that means is hosts really can't see the guest rating before they submit their rating. Same thing for guests. And also getting to a point of critical mass before you share the actual written feedback, because that is a part of it. You rate the host and again the guest from a star rating perspective, 1 to 5. And then you can also leave a review, which is written text and so really making sure that that can be anonymous if desired, and making sure that when you have too few reviews, that's not something that you can see until you get to a certain number.

Are there specific questions and how you design the questions that help people be more honest with their reviewer? Rating?

Yes. So we start off with just the star rating, and even just on the stars, we learn so much. So, for example, with each star, there's a word that accompanies that star. So with each star, there is essentially a word next to it. So, for example, five stars is great, four stars is good. And what's really interesting is that based on what culture you're from, giving even just three stars is very different by culture. So, for example, the French, when they see three stars, they think, this is a good experience. If it's good, I will give you three stars. Versus, in the US Anything less than five stars means something went wrong in some form. And so understanding, just like the cultural differences and understanding what some somebody's head goes to when they see the word great, you know, in the US great is, you know, most people should earn great, but, you know, for example, in certain cultures, great is a very rare thing. So even just that was something we had to really learn.

Ratings are hard, but they're especially challenging for international platforms like Airbnb. When we go on a listing, we would expect the rating to simply represent the quality of the listing. But can we take that for granted? As Jay Z pointed out, how people in different cultures respond to the same rating scale varies dramatically. Americans are very individualistic, so our culture encourages us to stay authentic and freely express our preferences. When responding to a rating scale, Americans are more likely to use the extreme ends of the scale, selecting great if we like it or terrible if we don't. However, people from more collectivistic cultures, Japan and China, are good examplestend to want to avoid conflict, which makes them less likely to give a strong like or dislike when they give a rating. So if they're asked to give a rating based on a scale of 1 to 5, they would select 3. More frequently, this response bias even occurs when people are simply answering questions about their own personality or values that have nothing to do with others. This means the rating that we see may not be an accurate reflection of the quality of a listing.

On the back end. Do you correct for that? Or how do you make it so that if someone in a different culture rates you poorly, that it doesn't take away your host status or your booking rate?

Yeah, so we do want to do a lot of guest education. So we essentially wanted to inform people, once you hit three stars, the questions that populate after that really make it clear that something. We believe that something went wrong, which is why you pick three stars. So we ask questions like, okay, well, what didn't meet your expectations? Right. So. So one is just the form that populates after someone picks a certain setting. And then we actually wanted to first just take the time to really see the data, to see if there were these nuances and differences. And then one of the things that the team started to do near the end was to factor that into the actual adjustment of the rating, but more importantly, into how things show up in search.

Cool. So we know that in behavioral science, how we ask the question determines a lot of how we answer. And I think one of the cleverest thing in the review system, or there's lots of clever things, but one thing that is outstanding is that you're asking people first to give the host personal feedback that won't be public, and then you ask them to write their public review. Was that intentional, where you're getting people to confess to the host changes that they'd recommend and then publicizing it for everyone to see?

Yes, it was intentional. And part of this is just the culture that we want to have on Airbnb, which is a culture of belonging and real candid guests and host interactions. And so we wanted to make sure that everyone felt like they were contributing to the community instead of ranting about something, because at the end of the day, this is someone's home. This is like a real personal thing inviting someone into your home. And so we did want to first start with what went wrong and how do we facilitate that feedback in a really thoughtful, yet candid way. And then we did want to give the space to be able to leave a public review, because that is an expectation. And it is also important for other guests who might be considering that place in the future to be able to see and decide.

That's great. It really allows somebody to vent in a way that is maybe more productive than the public system. And, you know, have the host potentially learn before it hurts their full reputation on. On Airbnb. So it. It feels quite clever, the design of that. Any other questions that you're proud of in the review system that have driven good behavior?

I think really thinking about the right length and when to really push for more, that was a thing that we considered. And what I mean by that is certain guests just don't like leaving reviews. Understanding that, and Factoring that in and essentially dynamically updating the flow to be able to say, okay, you know, you're someone who gives us a lot of data. We're going to ask you subcategory after subcategory, and we're going to ask you for color versus. We just really want you to give us a general rating and a public rating. Just also being attuned to what that guest's preference is so we can get the best results in terms of the rating.

Okay, let's switch to how hosts are measured. So Airbnb has a clever system by which hosts get not only reviews, but also something like a reliability or response rating. How did that come about? And what works and what doesn't work from the actual rating system that the host sees on how they're doing?

Yeah. So we started with what do guests need in order to actually make the right choice for themselves? We realized that just an overall rating wasn't enough. It was really important to see on certain dimensions, like cleanliness, like communication, how that host would actually be or how that home would actually be. And that's the motivation of actually creating that sub rating system.

Yeah. So if you think just about response time, this is a clever one. Can you talk about how this drives behavior? So if you're a host, you are pushed to respond quickly because you're measured on it. So maybe can you talk about the response time metric and how this comes alive for a host?

So there are a couple different pieces of response time. The first one is something that can just be automatically calculated, so we know on our side how long it takes for you to respond to a message. And then there's a more subjective piece of it of, like, just, was this host generally responsive to you? What is the guest's perception? And so we actually pulled both levers because both are so important to the guest experience. The first thing was that we would show essentially, like, the response rate, like, on the profile, and then we would also have that as a subcategory rating. So both were really important. And to the earlier point, you know, we picked the subcategories based on what guests want, but we also picked subcategories based on what we knew were things that we wanted our hosts to do. So we knew that a clean place meant a better guest experience. We knew that responsive hosts meant a better interaction. And so those were the things we surfaced because we knew that they would be important.

We are what we measure, and when we know we're being measured, we tend to overperform in One study, researchers gave some people pedometers to wear throughout the day. Those wearing the pedometers walked more than their peers without them. In another study, researchers measured students on how much they read. And guess what? Those who were measured read more than the control group. Airbnb is using the power of measurement to drive outcomes, and this is powerful. But the key to this strategy is to measure the right things. In another study with pedometers, and there are a lot people with pedometers lost less weight than people who didn't have them.

Why?

The researchers hypothesized that while people did walk more, we are what we measure. Walking may have crowded out other more intense exercises or just given us moral licensing to eat the cupcake. From my point of view, a lot of Airbnb's success has been thanks to to measuring hosts on outcomes that drive guest satisfaction, like response rate and response time.

Can hosts learn? So I know behavior change is hard, but if they get a low response rate or they have a cancellation, there's some penalty that comes with that. They have a lower public rating. They maybe can't post or can't have availability if they have cancellations. How good of this is actually teaching hosts to improve their behavior?

There are two really broad buckets. There are people who are craving that learning that feedback and are really hungry to get that information and then change. And then there are people who are mostly optimizing for the number of nights that they have booked. And so what you need to do for those second bucket of hosts is really tie the behavior to the end outcome of it's shifting bookings away. So if you're not going to be responsive, we're going to shift bookings away. And so really understanding which kind of hosts you're working with. And again, a lot of that came from just the engagement level on the dashboard, where people really clicking into some of their insights cards, where they're really clicking into the feedback they were getting on each stay. Those were the hosts that we really saw the biggest behavior changes from. So in that world, 100% hosts are not only able to change, but they're craving change. And then for the group of hosts that are more not necessarily change for change's sake or change for the guest experience sake, but change because it's the right thing. From a business perspective, you can also motivate those hosts by really tying the behaviors to the outcomes.

So it seems like there's some carrots and sticks. There's the stick of lower bookings if they don't behave Correctly. But there's also the carrot of being a super host and getting promoted and having it be a rare find. If you were to add any sticks or carrots, how would you, in hindsight, now that you've left and you have this beauty of hindsight, how would you change the carrots and sticks of the system?

Yeah, I think that one thing we did always struggle with was, what is the definition of a super host? And is a super host enough? Should there be more tiers than just that one? And so, you know, we worked on things like Airbnb plus where it wasn't just about your hospitality, but also just the quality and the design of your home. So really playing around with all the different ways of recognizing. So this is on the carrot side, all the ways to recognize a host and think about what are the ways beyond just hospitality, what could actually extend to their home that we can start to recognize them for so that we can actually have the right badging and the right incentives for everyone.

Badges can be a great cost effective way to boost morale and motivate positive behavior. We all love public recognition, sometimes even more than real monetary rewards. For example, one study found that software reps were willing to give up commissions of about $30,000 in order to qualify for an exclusive airline club. This honor earned them almost nothing financially, but represented a great deal in terms of recognition and status. This desire to be recognized can be motivating. Researchers in a field study found that when thank you cards were publicly awarded to the top three performers in small work groups, overall performance improved. This means remarkably improvement was even seen from people who did not receive the recognition. This may be because witnessing a colleague receive public praise compels other employees to improve their own performance to measure up.

So basically adding a little bit more status for different variables, adding more ladders that people can climb to get to the top.

Yes, because in the beginning, you know, superhost was something that was very difficult to achieve. But as the host community grew and as people just continued hosting them were amazing hosts. We want to acknowledge that amazing hosting and hospitality with more ways to incentivize.

What's your perspective on if hosts should be talking to guests more? One of the things a host could do in the many things that they have to do, you know, from cleaning to response time, is connect with the guest that's in their home. What should that look like for most hosts?

Yeah, communicating with your guests is really important. So making sure that you are responding quickly to any messages that they have, that's kind of the P0. And then after that, it does depend on the guests. So some guests really want that, like in person interaction. They want to have the host drop by and spend time with them, you know, actually walk them through things, but other guests actually don't. And it really depends on the type of vacation that you're taking and your preferences. And so we learned certain things. Again, more correlation. But it was helpful to then pass it over to our host. So, for example, if you're renting out a private room, one room within your home, you know, those guests typically are more happy and eager for more interaction. But if someone isn't renting out an entire place, they might be going with their entire family. And so do they really need that host interaction as much? Probably not, but they do want the to be responsive to whatever problems might arise.

Nice. So it's basically a read the room sort of thing. The host should be a little bit self aware enough to know what norm pool they're swimming in. Are they swimming in a chatty one or are they swimming in a leave me alone one?

Yes, always be responsive to questions, but then read the room.

Got it. And over time this is probably the answer. But in the beginning, there was more of a norm for conversing or chatting with the host because you were actually sharing. It felt like you were sharing your home. Any reflections on how Airbnb created that new norm? It now seems very normal to us, but in hindsight, it was crazy that people would come into your home.

If there are many people engaging in a particular behavior, others are likely to follow suit. This is especially true when people think the behavior is an evolving trend or that it will become increasingly popular. In a study, researchers tried to get people to use reusable mugs, telling people not just that others were doing it, but that it was part of a growing trend. They said more and more customers are switching from to go cups to a sustainable alternative. Be a part of this movement and choose a reusable mug. This framing increased the use of reusable alternatives by 17 compared to the control. People want to be a part of an evolving trend.

Any macro learnings on, like setting new norms?

I think the first learning is that you have to have that interaction, that chatter, that dialogue, in order to even feel comfortable inviting someone into your home. And so I think it was less that we created this new norm as much as the people who were willing to go out on a limb at the beginning and do something like that had to crave and want, want that kind of interaction. And so I Think it's more that than anything else. And then the question is, as we scaled, what did we want to maintain and scale with the platform versus what we're totally fine to be, like an early adopter kind of thing. And what we did realize is certain things like that feeling of hospitality, getting that handwritten note, or even just a note period, right? Like getting that really smooth check in. Those were the things that people actually cared about. Did they really need to sit down with their hosts for extended period of time? It depended on the person, like we just talked about. And also then bifurcating and saying, hey, maybe that should actually be an experience. Maybe someone, you know, who wants to be a tour guide or really spend that time bonding and showing that person around in the city, that can be something that you can access through Airbnb experiences and it doesn't have to be through homes. And so still enabling that interaction that makes Airbnb really special, but giving people the choice to choose it if they so desire.

That's really lovely. I've always kind of wondered why hosts don't do more async things like leave the cookie, leave the note, leave the bottle of wine. It seems like it's a free five star review if you're just do slight gestures of goodwill. Is that something that good hosts just intuitively know and you need to train other people to do or how do they learn to delight guests?

Yes. So people do do those things naturally and then we also taught people to do them. So for example, within our host dashboard, within our Insights cards, we would actually leave them tips around, hey, if you even just left a check in guide, we actually encourage every host to not only have one, but to print it out. You know, if you left a note, if you left a bottle of wine, if you left a treat that was local to the region, like that does really make someone feel at home. And so we would actually encourage hosts to do those behaviors that we saw great hosts do so that everyone would have access to that learning.

Have you found that in other PM roles that looking at basically the outliers who are exceptional can be taught to the normal people?

It's definitely a trick that we do all the time. So really looking at, you know, back in gaming, there's a term called whales with the folks who spent a ton of money, what was it that they bought? And how do we scale that down to someone who was getting started a hundred percent. Also within the people who use the most storage for something like Dropbox, what are they using their storage for? And how do we encourage the uploading of that type of file? Because people are finding value out of actually doing that. So looking at outliers is definitely something we do do. But I think the counterbalance is to think about, is this really just an outlier and cannot be generalized more broadly to the broader audience, or is it something that we can then teach the rest?

So maybe switching from what you learned from gaming, what are some lessons from Pocket Gems or your early gaming experience that you've taken with you in the rest of your PM career?

The first thing is that everyone wants to be delighted. So the more fun you can have doing anything, whether it's something routine or something that feels like work, the higher engagement you're going to have, the higher retention. And so that's something that gaming has left a very strong imprint on me around, which is delighting your user at all times.

So I love when we get experiment examples. Are there experiments at webflow, Dropbox or Pocket Gems that are worth that you could mention? Like, we tried A and B and we B one sort of thing.

One of the biggest things that influenced behavior change at Pocket Gems, specifically for the games that I was working on was really making sure that goals were apparent. And not just goals in terms of, hey, you have an achievement pop up and, you know, go do these things. But actually being able to scroll around on your map and seeing things in the distance. But I just thought this was really fascinating. So Paradise Cove, you open up the game, you're on an island, basically, right? So we're like, how do we get people to, like, feel like they want to do this game? So what we did is we literally covered the island with mist, so you can, like, see it, and you're like, oh, my God, what's under here? And then we had these things pop up in the mist, like these little structures, and when you tapped on them, it was like, you know, to find out more, you have to, like, actually get here, right? And so it spurred this. You know, farmville was very popular at the time, and farmville was like just like this clearing mechanism. You're like, oh, I'm going to clear the grass and, like, plant my corn. But it was very. It wasn't like. Like your goal was just to plant things, but you weren't trying to get somewhere. So we actually merged this concept of getting somewhere with this concept of this, like, behavior that you were doing kind of routinely that was like, very gratifying. And the combination of the two really, really worked. And so people were clearing and doing all the routine things, but they were really motivated to get to different statues, different areas in the mist on the island. So that's something I thought was just really fascinating.

Our need to discover is a powerful psychological driver. We often go out of our way or take great risks to fulfill our curiosity. This is because as humans, we have an innate desire to resolve our curiosity, even when it comes at a cost. In an experiment, researchers observed college students, students in a lab who thought they were waiting for an experiment to begin. The researchers left out some colored pens on the table and told the students the pens had been left out from a previous experiment. Students were told that green pens were innocuous, red pens would deliver an electric shock, and yellow pens might deliver a shock if clicked. Students clicked more of the yellow pens than both the certain shock pens and the certain no shock pens. Their desire to resolve their curiosity was so great that they were willing to endure painful electric shocks. Ouch. So, yes, JZ is right that having some uncertainty, some missing pieces, can be a great hook.

And then what was also really interesting was so we were like, wow, this is so motivating. But there's only one island and there are only, you know, 10 statues. And so one of the things that I'm probably most proud of introducing, because it's just so fun for our users, was we were trying to balance the fact that the game was loading really slowly because the more you added to the island, the slower it would perform, with the fact that people wanted to keep doing this behavior. And so one of the things I introduced was exploring new worlds. And so we essentially could allow you to explore Atlantis and all these other new places. And we essentially loaded the game state, swapped out all the assets, did a new storyline, had the same kind of, like, things hidden, but, you know, hidden behind kelp instead. And it was so exciting. Like, people, instead of churning out of our game, would just keep playing. They'd be like, I'm playing Paradise Cove, and now I'm playing, like, Atlantis. And now I'm playing all the other different versions that we launched.

Wow. I love that. So some level of novelty just keeps us going.

Yes.

It's a nice life lesson. Like, we need to create more worlds for us to entertain ourselves on a daily basis.

You need this familiarity, Right? So you're like, oh, I'm still clearing. And I know I'm like, I have the knowledge to be able to be successful in this world. But then you add in that novelty factor, and those two together is magic.

So when feature films were first being released, something Funny happened. And the movie angle, the way that the movie was filmed from a camera was actually stationary. So people filming just didn't move the camera. Why? Because it was crazy to move it. Back then you'd have photography, and in photography, you didn't move it. And now we look back and they just moved the camera. But at the time, they were taking the mental model of photography and dropping it into feature films, so it was hard to believe that you would have multiple angles. In hindsight, obviously very obvious. So we think, and we'll ask this about hosting in Airbnb World. If we think about that idea, what is going to, in 10 years, feel obvious to us about hosting or Airbnb that now just feels funny and maybe would be very weird if we introduced it.

I think the concept of whenever you are no longer using something, that that's an opportunity to enable your community to use it during that time. I think we're scratching the surface of it with Airbnb, where you say, hey, I have an extra room. Hey, I have an extra home. Maybe that's when I rent it out. But even getting down to, you know what I was talking about of, hey, it's Christmas break and I'm going to be gone for two weeks, how do I enable my community to use what I have? There's a spirit of giving. There's a spirit of optimization, too. Making the most of what you have. That is something I think will just be second nature in the future. I'm gone for even a week, it's going to be so easy to rent out my place or have someone else who could use this asset during the time I'm gone come in, use it. That's what I think the future could look like. So every single home on any single street could be an Airbnb for some period of time in the year.

I love that. So it's like maximizing our ability to share as a society, but making it wildly easier for us to do. And maybe we call it optimization versus sharing. Some more people do it.

Yes, the ultimate thing you achieve is sharing. But for every single single person, how they share and how you motivate them to share can look quite different.

That was Jay Z, a former head of product at Airbnb. We talk a lot about individual behavior change on this show, but Airbnb is a bit different. They're an example of group behavior change. Group behavior change is hard, and what Airbnb has done is impressive. Think about their core business. They've gotten us to willingly share our homes with strangers. There isn't one recipe for how to change societal norms. Even suggesting that there is a recipe should be debated, but I would argue that Airbnb leveraged one major tactic we can learn from, and it's something Jay Z was a big part the Review System A review system creates trust where none existed before. This is because reviews create a reputation system. In the Prisoner's Dilemma game, two members of a criminal organization are arrested and imprisoned. They can't talk to each other or collude on their story. Each person could rat the other one out, or they could choose to protect the other person in hopes the other person would do the same for them. And they both walk. So what happens in games where the participants play multiple games in a row rather than one and done? They tend to find a cooperative equilibrium. They both keep quiet and they both walk. In games where there is just one round, the players are more likely to wrap the other one out. A review system creates a long term repeated game for the hosts and the guests. They are pushed to behave better and cooperate because in the next round, if they don't, they'll get punished. Airbnb is leveraging this idea and it's arguably why they have been so successful. Hosts and guests are playing the long game with Airbnb and acting in the best interests of the platform, not just themselves. This is how Jay Z and Airbnb change the game and the norm. By creating a long game system like this, strangers are able to trust each other. The insight is not just that review systems facilitate this, but that companies in general may want to play the long game more to create new outcomes, new norms and new markets. The Science of Change podcast is presented by Setsale act on the moments to lead to a sale. Visit them at Setsale co or on LinkedIn to learn more about the world's number one revenue execution platform. If you like behavioral science and want to learn how you can apply it to your work, head to irrationallabs.com we have a variety of courses available. Use discount code podcast for 20% off this podcast is produced by Irrational Labs and StudioPod Media. Our executive producer is Rachel Roberts. All episodes are written by me, Kristen Berman and Ying Lin with expert script editing by Jack Buer. Special thanks to Lydia Troup for fact checking and citations.