Stanford Seminar: How a Facebook Designer Thinks - Julie Zhuo

So I know this is probably a huge cliche since probably every single speaker says this, but it's a huge honor to be here. I was sitting in your seats about 10 years ago, which seems like not that long and yet quite a long time. We just celebrated our My class, class of 06 just celebrated our 10 year reunion here a couple months ago, which is a lot of fun. So it's really, really great to be here. I remember when I was sitting in your seats, and I had just come from Texas, that's where I was raised. And I'd done some web design in the past, but I didn't really know anything about Silicon Valley. I didn't know anything about this kind of tech hub and what it means to have a career here and start startups and all of that prior to coming to Stanford. And I remember sitting in the audience and listening to the stories of people who had kind of come before me, and in particular Marissa Meyer and Jen Fitzpatrick. And they'd stand up here and they would talk about their adventures after leaving Stanford and all of the great things that they got to go and build and work on. And I was just like, wow, this is super, super exciting. And now I'm excited to be here talking to all of you guys and hopefully sharing a couple of fun stor along the way. So, as you guys heard, I was here. I did my computer science degree, my bachelor's and my master's, and I was also lucky enough to be part of the Mayfield Fellows program. So how many of you guys know about the Mayfield Fellows program? Okay, so for those of you who don't know, this is an awesome program that is really about diving into entrepreneurship. And so one part of the program is that between your junior, your senior year, you go and get an internship with a startup somewhere in the Valley. And along the way, as we're interning at our respective companies, we're learning about entrepreneurship, we're taking classes, we're doing case studies, and it's fabulous. It's awesome. It's taught by Tina. I had a lot of fun doing it. So at the time, the startup that I chose was actually Facebook. And this is a little cheating because in 2006, Facebook was already about 100 people. So it was a startup, but on the larger end of a startup. But it was something I was really excited to work on because it was a product that I and all of my classmates at the time used very religiously. So I went there and I remember my first day at Facebook. In fact, I was signed up to be an engineer and work on some of their photos features. And I had a mentor and her name is Ruchi. And on the first day she said, okay, there's been a change of plans, I'm switching jobs. I'm not going to be engineer anymore, I'm going to be a pm. I don't really know what that means for you, but let me introduce you to this pod of people over here. This is our design team. Why don't you sit with them and chat with them? I've got to go run. And so she left me in this pod with this group of designers. And before that, I'm not a designer by trade, I didn't really know anything about design. But I got to talking to these people and I sat at that pod and lo and behold, fast forward 10 years and now my career is in design. When I look back on it, it sort of seems really quite random how it happened. But for me, what was so exciting about design was the chance to work at the forefront and thinking about what are the people who are going through the experience that we're building, what are they feeling, what are they thinking, how are they able to understand the products that we built? And so to me, that was what was super, super fascinating about design. And it also wasn't that crazy because at the time the designers that we hired were also our front end engineering team. So it was a little bit of both designing and a little bit of coding. This is back in the world where we were still operating with just websites. So you just had to do some CSS and JavaScript and front end PHP. And it wasn't nearly as complicated of an ecosystem as it is today. But we got to building and one of the first things I worked on was photos and photo products. I also remember one of my first big launches a couple months later, which was Newsfeed back in September of 2006. Over the years, I've worked on things like the Facebook platform and ushering in a bunch of applications built on top of Facebook, including a lot of games. Back in the era of 2008 and 2009, I worked on Profile and Timeline and I worked on Newsfeed. And today the team that I lead, we work on design for all of the core features of the Facebook application. So when you go on Facebook and you want to catch up with your friends, you want to share, you want to watch videos or join a group, those are a lot of the things that my team works on. So all that said, that's a little bit of an intro into how I got here. The thing that I think has been the most fascinating for me as I've looked back on all of the things that I've worked on, is the moment when you're starting a project and you're thinking about, here's this awesome new idea that I have, and I can imagine how it's going to work and how people are going to use it, and I want to build it right now. And you rally a bunch of other people together with you, and you guys are all excited about this idea. That's how everything happens. That's how any idea ever sees the light of day. But the question that I found super fascinating is how do we know at that point in time whether this is an idea that's going to be successful? How do we know after it's all said and done and we've worked our nights and weekends and we've gotten it out to the world, Will it actually be something that people find valuable and that they'll find easy to use and that they'll find well crafted? And being at Facebook, we've had our share of features, some of them that have gone on to be great success and some of them that haven't. And a lot of times at the beginning, it feels like there's really no way to tell. It feels like you're kind of rolling the dice a little bit. And this is the thing that I wanted to really study and kind of reflect on over all of the different products that we built is what were the patterns for the things that were successful? Are there ways that we can tell as we are building whether this is something that's going to work or not? And what came out of that? And a lot of discussions and a lot of post mortems and a lot of just looking at the things that we had built was a framework of just three simple questions at Facebook that we now use to ask ourselves whether the things that we're building feel like that they're on track. And there are three very simple questions, because it wasn't going to be a manual that everybody was going to memorize and understand, but it's three questions that we wanted everyone at Facebook, no matter what their role was or what they worked on, whether they were engineers or designers or product managers, to keep in mind and, you know, when they're having a team meeting or whether we're reviewing the product or whether, you know, they're actually just talking with a colleague about an idea, to just think of these three questions and to ask them and to make sure that we have really, really good answers to these three questions. So that's what I'm going to talk with you guys about today. So the first question is the most basic and the question is what people problem are we trying to solve? And the key word here is really the word people because of course, whenever we build anything, we're trying to solve the problem. But what tends to happen is that you start to think in the mentality of your team or your company and you start to say things like, the problem we need to solve is that we need to optimize the click through rate of our page and you'll hear things like this all the time, boiled down in small ways and in large ways. And that's not a people problem. A people problem, as we define it is if you go out and you talk to someone on the street and they were to articulate a problem that they were having, that's how they would say it. That is the people problem statement. There's a couple things that we look at to make sure that this is a valid problem people problem statement. The first is that it needs to be human and straightforward. So we're not using words like ctr, we're not using words like optimize or you know, integrate. Like these are not words that people on the street would use. You know, these are not words that people who are outside of the tech community are going to use to talk about their problems. The second thing is we want to make sure that it's solutions agnostic. A lot of times we start problems by saying, I am going to build an app that blank or I'm going to design a website that blank. And already in that statement is an inkling of what the solution is. But what if app's not the right way? Or what if it's not supposed to be a website? So a good people problem statement gets away from trying to already constrain it into a particular solution. The third thing is that it shouldn't be about Facebook or your company or your team or whatever winning. I think a classic example is if you say our service is going to be the best at, blah blah, blah, a person on the street doesn't care if your service is the one that is the best at that. They just want to know, hey, for this problem that I have, what is the best solution? Stating any problem as like, we, you know, our team, our company, whatever wants to win is, is not getting close to, to the things that people actually want. The fourth thing is that it gets at the why? So, you know, sometimes you might have a problem like, people aren't, you know, discovering this page. And that is a problem that we need to solve. But it doesn't go one layer deeper. It doesn't get at, well, why, you know, what. What's actually the root cause of why people aren't discovering this page? Is it too hidd? Is it too confusing for them to find? Like, it's got to get to not just what is happening, but what is the reason why it's happening. And finally, a good people problem statement can not just solve functional problems like a confusing flow, but could also get at emotional or social problems. Sometimes people just want to feel like they belong. Sometimes people just want to feel like they're validated. Those are things that also constitute, you know, something that, that, that when you ask people, they might. They might say. So let's look at some examples. So earlier this year, I had the privilege of attending the moms 2.0 summit. And this is a gathering of a lot of influential bloggers and journalists who, you know, are mothers and who talk about, you know, parenthood and those topics. And so I got a chance to meet a lot of the attendees and be on a panel with some of them. And one of the things that I heard over and over again was basically this. And this is an example to me of a people problem statement. I want to talk about an interest with other people who are also interested but don't know where to find them. And this is especially relevant at this conference because a lot of times what would happen is you're a new mom and baby. You're in your friend group. Nobody else is having babies around the same time as you. But at the same time, there's so many things that goes through your mind. You want to know, is this normal? My baby just did this. Should I be worried? What are some resources? You want to have a community of other new moms to be able to talk to. And so a lot of times women will not know exactly where to find that community. And if they're lucky, friends or other people will say, hey, there's a local, you know, group here, or there's like a neighborhood moms group. But this is something that a lot of people in the audience were talking about as something that was a problem to them. So this is an example of a people problem statement. Here's another example. This one we've heard over and over again ever since we introduced the like button on Facebook. Everybody would ask, why don't you guys have a dislike Button, and that would be one of the top most requests that we've had for years and years. But, you know, that's not really a people problem statement. That's sort of just a suggestion. If we dig deeper, the reason why people wanted a dislike button is frankly, because not everything in news feed is likable. You know, people write about hard times that they're going through, tragedies that are happening in their lives. You know, recently we just went through this election, which was very, very charged. And, you know, people would read things or say things that made them feel a lot of different emotions, not just happy ones. And so that is basically the people problem statement. Not everything in feed is likable. And I want to be able to easily express other things. And finally, one more example. So sometimes I call these people problem statements. Sometimes they're not just problems in the way of like, hey, is someone going to bring this up as a hardship in their life that they want solved? Sometimes it's just more of an opportunity or a thing that if you ask people if they wanted, they would say yes. An example here is the desire to share spontaneously and authentically. A lot of times we're going through a pretty cool experience and we might capture it at that moment and then an hour later go and upload it on Facebook. But how cool would it be to actually have that experience live with our friends and our family? So that is an example of another people problem statement. So those are basically, you know, the first. The first thing that we always ask is, okay, do we have like a statement that is really about what problem we're trying to solve for people? That's the first question. So now the second question is, well, how do we know this is a real problem? And by know, I mean, well, what evidence do we have, right? Is there qualitative evidence that it is? Is there quantity, quantitative evidence that we can look at? And I highlight the word real, not because I think that there are like fake problems, but just because I think the question to ask is, is this a problem that is worth solving? All of us have limited time, energy, resources, money, whatever, to be able to devote our being to trying to solve problems. What makes this one the one that we should pick out of the thousands or millions of problems that are out there? And so this question is really about just making sure that the opportunity is something that is worth tackling and that we aren't just, you know, solving problems for ourselves individually, but that we actually are very aware of the problems that the audience that we're Building for is facing. So for the example of groups, you know, I want to talk about, I want to find other people who are interested in discussing the same things as I am. The way that we tried to validate whether or not this was a big enough use case was, of course, we went and we talked to a lot of people, but we also looked at some of the data that we had. So when we designed the group's product to start with, we imagined that most of the time, you would get invited to the group. So I might start a book club, and I'm going to invite some of my friends who like reading the same books as me, and that's how we're all going to know about the group. So in our initial formulation of groups, it was really all about being invited by somebody else who was already a part of this group. But one of the things actually that some of the engineers on the groups team had hacked over the years was this little unit called groups you should join. And so some of you guys might have seen it. Sometimes you are scrolling through your feed and this little story will pop up, and based on what your friends have joined and groups in your community and groups that are similar to things that we think you might be interested in, you'll see some recommendations, and you can go and explore them and decide to join them if you want. And we thought that this was just a small feature, but really most of the inviting was going to happen through the normal invite process. But when we took a look at the data, what we found was that actually a third of group memberships were starting to happen via this little groups you should join unit, which meant that there actually was a need and a desire already, using the tools that we currently had to, that people wanted to find things that they weren't explicitly being invited to, that there were examples of all sorts of things that they wanted to be able to participate in and see that they just weren't getting connected to any other way. So this was, I think, good evidence for us that if we wanted to devote something, a larger solution to helping people discover groups, that that was a worthwhile endeavor. We also talked to a lot of people in a lot of different groups. One of the folks we talked to referenced this group called the physician moms group. This is actually one of the most active groups on Facebook. And I think something like one out of four doctors who are mothers in America are a part of this group. And one of my friends actually happens to be a doctor who just had a baby, and so she was lucky Enough to have one of her colleagues add her to this group. But she talked about how meaningful it was to her. And she would spend hours on this group every single week because these doctor moms would be sharing tips on how to balance their work and how to really tackle a lot of the issues that come from being in that profession. And this is an example of a group that everyone who is a part of it finds it hugely meaningful. But not everyone knows about it, because how would you know that this thing even existed out in the world unless somebody told you? Another example is I recently attended our 10 year reunion. And at the reunion, I think a lot of people were using groups to coordinate events and meetups. And the biggest barrier is that half of the people that I talked to didn't know that these groups existed, that there was a class of 2006 group and that there was a reunion group, and that there were groups for different dorms who were planning their get togethers. And finally, there are a lot of groups. I'm a designer. And one of the top groups that a lot of designers in the community join is this group called Designers Guild. And a lot of designers share tools and articles that they've read and other resources. And every time I happen to add someone because I thought about them, that they might enjoy it, they're always like, wow, I didn't know about this. That's the first thing that everyone always says, I didn't know about this. So all of this, when we did our focus groups and our research, that was more evidence that in fact we should be doing more to help people discover groups. So we ended up doing was actually building a pretty simple Discover feature. If you go to your groups tab, right next to it, there will be a tab called Discover. And we did our research on what were the topics that people were most interested in. So you can scroll through, you'll see suggestions based on things that you already are a part of. But then you'll also be able to go and browse by category. And this is where you can find parenting groups if you're a photographer, People to go on photo trips with on the weekends, or play sports with, or joining a writing group. And these are all groups that are public and that actually want to welcome new members for the people. Problem of not everything that I see in newsfeed is likable and I want to be able to express other emotions. We wanted to make sure that this was actually a problem that many people faced and that the solution that we had would actually directly address it. So we talked to a lot of users. We had them go through their feeds and describe to us for each story, what was their reaction, what were they feeling? Sort of like free flow. Tell us what's going through their minds as they're going through their feeds. And a lot of times people will say, well, there should be more ways for me to just say something. Because what I like about the like button is that it's so simple. I don't have to go and comment and the keyboard comes up and I have to two handedly type something. I like the fact that I can just in one gesture, kind of scroll through things and then say that I like it. But the only thing I can do is like, and there's got to be other ways for me to express other emotions. We also looked at how are people using, how are people expressing that they don't like something today? And we looked at the stickers that people were using and leaving us comments as well as the emoji. And what were kind of the most popular ones. How frequently were people just leaving a simple sticker or a simple emoji? And finally, we looked at also short comments. So we looked at how many comments were just one or two words, and all they expressed was like awesome or like that sucks or a very, very short phrase that we felt we could encapsulate. So with all of that, we decided, we looked at all that data, we looked at what were the most common emotions that people wanted to express. And we designed something that we felt was sort of fulfilled that criteria of being really, really lightweight. So you can still do it with one hand and one gesture. It's not like multiple taps. And we also, you know, taking all of the data that we had gotten from what were the most common things that people, you know, the common reactions that they had, we built the reactions product. And so you just scrub through and we took the top, most, what we wanted to be universal reactions. And that's the options that you see in that tray. And finally, for the people problem of I want to share spontaneously, authentically in the moment. This was interesting to us because we had the Facebook Live product out for celebrities, and we built it for celebrities and public figures first. Because when we talked to them, that was something that they told us very directly that they wanted. And it was something that they were used to because they lived their lives in the spotlight and they're very used to going onto the red carpet and doing interviews and being broadcast live. And it was something that they felt they could do to kind of connect with their fans a little bit more. So this is something that we had out that was only available to public figures. But the interesting insight for us was, well, is this something that, that, that people want who aren't public figures? You know, is this something that people in the audience, you guys, me, you know, all of us, would this be something that that non public figures would use and find valuable? And that is the question that we set out to try and, and uncover and prove. And actually one of the easiest ways for us to do that was to, you know, just build it for ourselves. You know, we already had the code that ran for, and we just turned it on and see if anybody at the office uses it and what their reactions were. And we saw actually a lot of really creative ways in which people used it. Some people used it to broadcast their team meetings for people who were working from home. A lot of times when our VC programs weren't working super well. The other benefit is that you record the video and then it's available afterwards. So for people who couldn't tune in live, they still had a chance to watch the contents of the meeting. We also saw people just take it out for a weekend. So they'd go to brunch on Saturday with their family and then they would go live and then people would chime in and there would be this conversation. And the people who did this were like, yeah, it was awesome. I had a great time. And then we also saw people taking it for a run. So a team at Facebook was doing this relay race and they wanted other folks to kind of cheer them on and give them support. So they went live. And this is also a super fun thing for them and for all the people who participated. So that was some inkling that, hey, this could actually be something that other people wanted. And so what we did was we launched it to a small percentage of our users as kind of a test to see if there was interest in the market. And that was the Facebook Live product on Android. It launched on Android first, and right away we did see a lot of really, really creative uses. So one woman who was a tattoo artist started to broadcast live as she was in the process of tattooing somebody. We also saw a woman who broadcasted her wedding live because she had family members who couldn't make it and who couldn't travel the long distance. And it was really, really cool to be able to see how she could engage and interact with those audience members and almost have it feel like they were a part of that experience. And I had a Lot of fun taking it out. I managed to get reservations to one of the best restaurants in San Francisco, Lazy Bear. And it's a really cool experience because the chef stands up there and he explains every single dish, and he kind of talks about all of the love and care that went into how this dish came together. And I just decided to go live, and a ton of my friends tuned in, and everybody's like, I really want to go to this restaurant, too. So that was really fun. Okay, so that brings us to the last question. So let's say we built this. We validated that it's something that people want, or we think there's enough reason to believe that it's, you know, something that is a real problem, and then that's something that is worth solving. So now we build it and we put it out there. But even actually before the thing is out in the world, in fact, even before maybe we have a full prototype, the question for us to ask is, well, how will we know if we solve this problem? You know, like, what would the. What would be different in the world? What would, you know, if we fast forward and now this thing that we've done is out there? How will we know if we should be happy, if we should be excited, if we think it's didn't live up to our expectations? What exactly should our expectations be? And I think this is a really, really important question to ask up front. Too often, what ends up happening is we have this idea, and then we build it and we launch it into the world, and then results come in. We're looking at the dashboards, we're looking at how many people are downloading it and what they're saying. And, you know, there's tons of data points that we're trying to, you know, interpret and put together, but it's hard at that point in time to then be very, very objective about did we solve the problem? Because a lot of, you know, our natural inclination is to read into the good things that people are saying and to kind of consider all of that effort to have been worth it, right? And that there's biases that kind of come from when, you know, you're already looking at data and you're trying to interpret whether or not it's good or bad. It's much better before you launch, to have figured out, you know, what constitutes success for you so that you can, you know, go into the launch with that understanding. And as results come in, you can map it to your previous. You know, this was my criteria for whether or not we've solved the problem or whether this was successful. And so what we do here is we want to make sure that we set measurable goals and metrics, and measurable is really key. Measurable doesn't always mean, like numbers or data, but it does mean, you know, there's a criteria where if I did this thing and I got this result, I know what to make of that result. So for the example of groups, you know, we wanted to help people solve the problem of helping them find, you know, other people to talk about their interests with. We ended up building a Groups Discover dashboard. What does success look like for us? So before we launched, we determined that, well, if we were successful at actually helping people solve this problem, we would see that more people are then discovering groups that they're interested in and joining them. But not just joining, because I could make a giant button flash and make it red and I'm sure more people would click on it and more people would join. That doesn't really count. What really matters is that they join these groups and these groups are actually meaningful to them. Which means that if we fast forward a couple months, three months, let's say, and then we look back, these people are still using those groups and they're actively engaging, meaning they're talking with other people, they're reading the content, they're sharing content they're liking, they're commenting. We want to know that people are joining groups that are actually valuable for them and that they're spending time on. So that is actually the more important metric was not just joins, but meaningful joins. For the example of, you know, I want to be able to quickly express my feelings about a story. But not everything is likable. You know, we launched reactions, and prior to launching reactions, we wanted to make sure we measured a couple things. If we were successful here, then what we should see is that, well, people are using these reactions and not just using them. It's not good enough if all the people that were previously using a sticker or an emoji or a short comment then converted to using Reactions, because then it's just a one for one trade. If reactions was actually more lightweight, we should see more people using that than the previous set of people who were using all of these other tools. So that was really important to us. So that was kind of the first thing that we looked at. The second thing we looked at was we wanted to make sure that every reaction did get a fair amount of usage across different markets. And this was important to us because in the beginning, even when we started to design Reactions, we had considered lots of different options. We'd considered maybe it'd be funny if you were like an awesome button, or there's tons of different things that we could have put in order to make it more expressive. And the reason why we distilled it back down to six is because we wanted to make sure that it was simple to use. And if you were going through and choosing from a grid of 20, suddenly it doesn't feel that lightweight anymore. It feels like you're pulling up an interface and you're choosing something and it's going to take more than a couple taps to get you to what you want. So it's really important that we nailed and got to a really, really small set, but that we picked the right set. So we wanted to make sure that these were universal. And it wasn't like people in certain countries were using certain ones more, or people in certain demographics were favoring others because it was a more niche thing. We wanted to make sure that these were actually things that expressed the emotions people wanted to express in all of our different countries and in all demographics. And then finally, we wanted to make sure that the experience was good for the receiver. So if you posted something and then you got a bunch of reactions, did that make you feel bad? Did that make you feel confused? Was it like, did you not know what was going on? Or did you just feel like it was a negative experience and it made you want to share less in the future? That was also really important. And that was actually one of the reasons why we never straight up put a dislike button, because dislike in of itself can be very ambiguous. Am I disliking the content that you posted? Am I disliking you as a person? Am I just not agreeing with your point of view? And so in all of our research and understanding of dislike, we knew that it was something that people were going to get confused by and wasn't going to be a great experience. So when we launched reactions, we similarly wanted to make sure that that wasn't the case, that the emotions that were being expressed felt like they were adding to the poster's understanding of what their friends and audience had to say. And finally, for the example of I want to spontaneously and authentically share in the moment, what we decided to measure and look at was that of course people were broadcasting. But again, it's not just did they broadcast once, because that's something that you can make happen by doing a ton of promotion, and by putting it front and center, people will Try a lot of things once because they don't know what it is or they think it sounds exciting. The true test for us was when they went live and they had that experience, did they like it enough to go and do it again? That is the key that tells us we built something that is valuable and that's worthwhile because people will come back and do it again and do it again and they'll incorporate it and make it a part of their lives. The second thing we looked at is we wanted to make sure that, well, maybe it'd be fine if broadcasters loved it. Maybe everybody wants to go live all the time, but is it really a good product if nobody wants to watch what people are broadcasting live? Right. And then it's sort of one sided and over time broadcasters are not going to want to do that because nobody ever watches anything. We wanted to make sure that the content was actually interesting and a way that we could measure that was is the time spent watching live videos going up proportionally and you know, when we run surveys and other things, are people saying that this is like a valuable thing that they want to watch? So to recap, I went over kind of the three questions that we always ask in product reviews. And you know, anytime we're talking about products, we want to make sure that we know what people problem we're trying to solve. That it's very clearly defined, that we can picture, you know, the audience or a particular person in mind as we're designing the product and that we're not solving problems for ourselves or our company or our team, but that we're actually focused on an audience and a problem that matters for them. The second thing is just making sure that we know it's a real problem through looking at whatever quantitative data we have, through doing focus groups and talking to different users, understanding that this is something that is a good use of time for us to work on and to solve and finally being very, very rigorous about what does success look like and how will we know if we put something out there and it did what we wanted it to do and if not, then we go back to the drawing board, we learn what didn't work and we keep iterating. But these three questions have helped keep our teams focused on what matters, which is ultimately the people that we design for. So that's it for this presentation and I think we've got plenty of time for questions. Yes, over there.

So there are studies that show that the use of Facebook is often linked to low self esteem and sometimes depressed feelings of People. So I was wondering, from like a product design standpoint, is your team doing anything to like address some of these negatives?

Yes. So the question was, you know, studies. Do you know which studies in particular I'm looking at?

And they haven't mentioned any in particular, but they say like Journal of Social and Clinical Psychology has made publications on.

So the question is, does Facebook lead to people being lonely or lower self esteem? And what are we doing to address that in the product? So this is something that I think is really interesting to us because we hear a lot of concerns around, is the product good for people? Is it something that's helpful or is it taking away from them? And all the studies that we've been able to do and that we partnered with a lot of institutions to work on, I think it goes both ways. So there's a lot of studies that sort of show that it's actually better and that it helps connect you with weak ties, which helps you get more of an understanding of diversity and other things in the world and other times where people who go through difficult experiences are able to get support from the people in their network. Certainly I think that there's also things where maybe some of you guys have experienced the idea of fear, of missing out, where you see other people having these great experiences and it makes you wonder about your life. So all in all, I don't think there's conclusive evidence and this is something that we have studied where it's either drastically better for the world or drastically worse for the world. What we do believe in is that by being connected to people and by being able to share with the world that that is something that is worth doing and that's a human and a universal need. And that's something that we believe very strongly in. In terms of other things that we have done, there are problems on Facebook that we know are very difficult. Things like bullying or even just more commonly breakups. These are things where because of the, the fact that people are using Facebook, especially for things like breakups, you know, you used to, you know, there's tons of photos of you and this other person and there's, you know, you're constantly seeing posts from this other person. Because when you guys were together in a relationship, you know, we knew that this was an important person to you. And after a breakup, which is a one time event, you know, what we heard from a lot of people is that that's actually a terrible experience. Right. You don't want to go into Facebook and see newsfeed stories from this person, a lot of times you don't want to see photos of this person. You actually, in certain cases, want to have any interaction with those people. And that is actually an example of a problem that we thought was really meaningful and something worth addressing, even though it doesn't affect everyone in the room every day. It was something that, when it happens, is difficult enough that we wanted to be able to put a solution around. So we launched a feature called Breakup Checkup earlier this year. So after you say you're no longer in a relationship with somebody, we want to make you aware of the tools that are available for you to control your content and control your privacy with that person. So that's an example of something where, looking at the common things that people go through that are just a part of life, how can we make that a little bit better and a little bit easier on Facebook? I think bullying is another one that we care very, very deeply about and want to make sure that we have the right tools to immediately be able to flag and to take down and to also make you aware of your options to block people so that you don't have to keep interacting with them on Facebook.

What has been your favorite product you've been a part of designing?

What has been my favorite product that I have designed at Facebook? I think that's a really, really difficult one. I think for me, it's probably Newsfeed. And the reason why I think Newsfeed is so interesting has, I think, just a really interesting origin story, which is that when it launched, some of you may be aware of this, but it was actually one of the least popular products ever. And this was back in 2006. I'd only been at Facebook for a couple months. And the origin for Newsfeed is that for all of us who are using Facebook every day, we were following the same pattern. We would go open Facebook and we would check the profiles of our 10 best friends. So we'd go and see, what did Jane do today? Did she add anything new? Ok, no. Let's go, Sam. Did he add any photos? And this was our pattern every single day. And that was a little bit inefficient. So we thought, well, what if we didn't make you look anywhere? What if you just landed on Facebook and we told you if Jane had updated her photos and if Sam had changed what his favorite band was, and we'll just save everyone a lot of work. And so we were very, very excited about this feature. We loved it. We thought it was amazing, and we put it out there and people hated it. I can't even describe how much at the time Facebook was a college and high school site. And I think about 10% joined a group called I effin hate Facebook Newsfeed. And at the time Facebook wasn't big news outlets like CNN or major media outlets never wrote about Facebook because it was just a college site. But at that moment it was such a funny story that so many people hated this feature that they started reporting on that just like, ha ha, there's a site out there that a lot of college kids use. And. And then they launched this feature and like, oh my gosh, was there a huge uprising? And they did a terrible job. So that was our first major wave of press from major media outlets. But what was interesting is that for us, we started to look at, well, did we really mess up? Is this something that's actually truly awful? Is everything that we believed in wrong? And the things that were interesting is that even though people joined these groups and told us all of these things that they didn't like they were using it, they were using Newsfeed. The only reason people even knew about the group I effinhate Facebook Newsfeed is because it showed up in their newsfeed and then they went and checked it out and then they joined it. So we listened to what were the actual concerns. And a lot of the concerns were around, I want to have greater controls around which things do get published into Newsfeed and which things don't. So we heard those concerns and we made those modifications to our feature. But it was because we were looking at the signs that there were some things that were working, there were things that we wanted to improve as well that led us to not just completely scrap the product despite the backlash. And what I think has also been so interesting about Newsfeed for me is that it's not just a one time feature. It's not just like we put it out there and we're like, cool, good, good job, we're done. Newsfeed is probably one of the most iterated on products over the years and everything from the ranking and the algorithm and how we present information and the design of stories, especially as cell phones got better and photos got more high res and now we were able to take videos. So to me, what I've loved about that process is it's an evolution and it's never something that kind of stays still and we can say it's done. It's that we're constantly thinking about and we're constantly Trying to make better and better. A question over there. Yes, I worked on my question a bit long. I'm Ash.

I'm a grad student here. I'm interested in democracy and tech. So one of the norms really vital to functioning democracy is the commitment to reason, facts and analysis. And recently, Facebook has received a lot of criticism regarding the proliferation of echo chambers and the misinformation. What is your perspective on Facebook's role in the election, in politics, and in our democracy at large? And second, how are you going to fight echo chambers and misinformation on Facebook?

Yeah, so this is, you know, I think with the recent election, I think it's been a pretty charged time. A lot of people have expressed a lot of. Can you repeat the question? Oh, sorry, yes, I will repeat the question. So the question is around echo chambers and filter bubbles, and what is Facebook's role in that and what do we want to do about it going forward? Is that right? Okay. So I think given the election, which I'm sure is something that a lot of us in this room are thinking about, we know that it's been a really emotional time for a lot of people and that there's a lot of different thoughts and emotions and other things that people are feeling around what has led to the state that we're in and how can we move forward. Mark had a statement about this that he shared last week and that of all of the news on Facebook, we do know that a very, very small percent, less than 1%, is what would be considered fake or misleading news. That said, we don't think any of that content should be on Facebook because that's not helpful for people. Nobody wants to go and read a story that is obviously a hoax or is spamming. So that's something that is important for us and that we will continue to work on. That said, one of the things that Mark also talked about was just how important it is that, you know, even despite this time where there's a lot of emotions and sort of charged debate going on that, you know, it's not. Facebook is not. We have to be very, very careful about whether or not we are censoring anything or trying to push a particular point of view, and I think this is a, you know, a super gray area that's very important. It's not. We don't want to be a place where, you know, people feel like there's a particular point of view that is being pushed on in Facebook, and that's really, really critical to us. And that's something that we are looking at deeply and wanting to make sure that we're taking very careful steps forward. Question in the middle over here. Yeah.

So Facebook has been investing a lot.

In Oculus and virtual reality.

I was just wondering what your opinion.

Of what problem this is trying to solve. The question is around virtual reality, and our invest in Facebook's been working a lot in virtual reality. What problem is that trying to solve? So I can't speak to this a ton because I don't work on a lot of the virtual reality products. But in the past, Mark has talked about his excitement for virtual reality as a new medium that he thinks will enable people to be able to communicate and share and is in line with our mission of making the world more open and connected. And the way that we think about it is when we started Facebook, the medium that we were all using was the desktop website. Mobile phones weren't a big thing. Everybody was using Facebook on desktop. But as technology shifts and as it was really clear that mobile phones were here to stay and they were efficient and we could do a lot of things on them that we had to adapt Facebook, our site and our services to the mobile phone. And in a lot of ways in the future, I think virtual reality as a new medium is really, really exciting. There's tons of things that we'll be able to explore and design and think about in that medium that we weren't able to do with desktop computers or with mobile phones. But all of the things that we're going to do there are, I think, going to be, in the same vein, aligned with the mission of trying to help people be connected and to be able to share. Question in the back. Yes. Can you share some examples in which a problem or an idea passed the first test, like you could not quantify it or you could not figure out how to measure and hence it did not work? Yes. Are there any examples of the questions. Are there any examples of problems we've looked at that have passed the first test? So it's a real people problem, but that we didn't have enough evidence was a real enough problem for us to go and tackle? I think that, yeah, absolutely. In fact, a lot more of that than actually the things that we end up building. I think we have this program at Facebook that a lot of people participate in, which is called hackathons. The idea is that we'll run these every six weeks or so, and everyone at the company who has an idea that they're excited about and want to hack on, they spend the night. It's usually one night or perhaps one night in one day and they form teams with other people at Facebook and then they just build that product. And then, and then we have this big thing called the Hackathon Fair where a lot of people kind of weigh in and there's judges and people present their products and then we figure out whether these are things that we actually want to kick off and build or to test or to actually make available on the products. And I think there's a lot of projects usually start around the idea of like, hey, I would like this feature. I would like to see. Here's one example of something that kind of came about a long time ago. I would love to see really immersive large photos for Facebook. I have a really large screen and I've got a lot of computing power and all the Facebook stories look really small. So let's try and make everything a little bit richer and a little bit more visual and let's do that. And so we actually tried that, we built that. But what we learned was that that's true for us here in Silicon Valley. We have the top of the line machines and we are able to have really fast broadband access so we can load really, really big images. But that just wasn't true in many, many parts of the world. So when we did that, we actually made the experience of people in those areas worse because they had to wait longer and things just didn't feel fit as well and it wasn't a great experience for them. So that's something that we end up not shipping because it didn't sort of satisfy the like, was this a big problem for everyone? All right, last question. Question over here in the middle. We talked in your introduction about interning at Facebook with over 100 people. What practical advice do you have for us when evaluating job offers for startups? Sorry, I didn't catch the last part of the question. What practical advice do you have for us when we're evaluating job offers for startups? So the question is, what practical advice do I have for people who are evaluating job offers for startups? I think the most important thing is whether or not this is a group of people that you love to work with and that you can see yourself being able to learn from and work with and they share the values that you care about the most. I think that that's the first thing I would look at and I think the second thing I would look at is whether or not the thing that the company is trying to do resonates with what you care about. So every startup out there, every company out there is trying to solve a particular set of problems, and some of these problems are going to be more meaningful for you than others. And so I think those were the two questions that I don't ask. Do you feel like this is something that you would be excited to wake up every day and work on because it's doing something that you believe in, and are you going to be able to do it with the people that you really respect and look up to and who feel the same way about you? Cool. Thank you, guys. Thank you very much.