Ellie Powers on Secrets Every PM Should Know About Android and Google Play | #ProductSF

At Product sf, Google Group Product manager Ellie Powers tells us some of the secrets every product manager should know about Android and Google Play. Many PMs and developers don't know that Google offers a lot of design and testing support. And Ellie shares a breakdown on how we can test and ship high quality Android apps. This presentation was recorded at Product sf, an event hosted by Greylock Partners that brings together founders, PMs and product leaders to talk about the challenges of building new, new, innovative products that change lives and create habits. For more podcasts, please visit news.greylock.com for.

Our next speaker, I'm really excited to introduce Ellie Powers. Ellie's a group Product Manager at Google. She's been working on the Android developer ecosystem, the Google Play ecosystem, for a really long time. She understands kind of, I'll bet you almost every out here has iPhones, but she understands how the other, you know, 90% or whatever, 70% of the world actually lives and uses products. And she has a real passion for kind of making things that make life better for developers and then ultimately make life better for users. So I'm super excited to welcome Ellie and have her kind of talk about what you guys should be doing on Android and how the rest of the world uses their phones. So thanks, Ellie.

All right, so as Josh mentioned, I'm Ellie and I'm really excited about how Android has grown. I'm one of those really weird people who is actually from the Bay Area. I was born about 5 miles from Google and I actually went to high school here in San Francisco. And the reason I bring this up is this is why I'm so excited about technology. You know, when I was a kid, I had a computer, I had a dial up modem and I thought that everyone was going to be connected just like me with a dial up modem. And it's been amazing to be part of this journey on Android and Google Play, actually seeing everyone in the world have the opportunity to get access to information, chat with their friends and anything else that they want to do. But it's happened through Android, right? It didn't happen in that other way that I thought it would when I was a kid. And what makes a smartphone great is not just the phone, but it's all the apps that all of you build and put on there. And when I started on the Android team about five years ago, companies would come to us and say, well, I don't know if I need to build an Android app, is that really necessary? I have an iPhone. It seems to work pretty well. For me, I don't know. And no one ever says that to us anymore. And it's been amazing to watch this ecosystem really grow. And one of the things I've been so lucky to be able to do as part of this team at Google is to go out and meet with folks like you, people who are product managers or developers working on different apps and games all over the world. And so what I wanted to share was the types of things that I feel that everyone who works on Android apps should know, but oftentimes I'm surprised to find that they don't. So there may be some things here that are new to you, but I really hope that everything I'm going to tell you today is very boring. Because if it's boring to you, it means that we've actually managed to get this information out there in the world. But I talked to Jos and he thought there's actually a lot of things where people need to know more. So I'm excited to share with you a lot of these kind of fundamental things about what Android is. So let's do a little bit of a poll. By the way, I imported my slides from Google Sheets or Google Slides, so I'm rocking a serif font today because, you know, I had a cool font. So does everyone have a mobile phone with them today? Probably. So who here has an Android phone? All right, cool. That's actually more than I expected. Who does not work at Google and has an Android phone with them today? Cool. All right, and how about iPhones? Does anyone here have an iPhone with them today? Okay, yep, that's pretty typical. And how about another smartphone? Do we have any maybe Windows Phone fan people in here? Maybe any trio users? Okay, and is anyone kicking it super old school with a feature phone? I have one friend who works at Google who actually has one, but I guess she's not here today. And how about any no phone users? Maybe people who've really thought that this addiction is just time to. Okay, I didn't think so. We're all in this together. Cool. So one thing that's really important to know about is here, obviously we live in this bubble and so almost everyone here has an iPhone. There are a few of us outliers, and that is really not typical when we look at a lot of the data. And I encourage you to go out and Google it. The vast majority of phone sales that are out there today, whether it's in the US or even more so in other markets, they're Android devices. And so as a product Manager. It really behooves all of us to understand our users. So Android has been so successful globally because it's so flexible. It's because of this flexibility that we're actually able to make the perfect phone for almost anybody. Right? So if you have $700 to spend, or. Or if you have $30 to spend, there's going to be an Android phone out there that's great for you. And this is amazing. Again, this is why Android has grown so much. This is why so many people have been able to connect. This is why, when you look at important situations, like let's look at refugee situations, one of the first resources that they bring to those camps is actually the ability to charge and reconnect smartphones. This is such an essential technology for people in all parts of life. And so we're really happy about this, but it also leads to a lot of challenges for some of us as product managers. So I want to walk through a lot of the different things that you need to know in order to make sure that you understand how to take the best out of Android. So, first of all, we're going to talk about reach. In fact, that's usually the number one reason why people are bringing their apps to Android. So there's been some talk today about mobile eating the world. My colleague before showed a very cute graphic of phones eating the world. And so we have some data from Benedict Evans here. You know, this is really the first universal tech product, right? Not everyone in the world got to have the chance to have a television. To have a tv. Someone mentioned the car. Not everyone in the world has had a car. But we are rapidly approaching a situation where every single person in the world almost in the next few years is going to have a smartphone. And this is unprecedented. And so if you're a product manager, you know, we talked earlier before with Reid, talking about the distribution. Now, thanks to Google Play and other distribution networks, it's really possible to get your product out there globally. But, you know, this is not only an opportunity, but also there's a lot of challenges there because different people all over the world are going to have different needs, and that's important. And so here's what the reach looks like. Today we have over 1.4 billion active Android devices out there. We have partnerships with a huge number of operators and carriers, and it's a massive audience. So if you want to have that universal product, you definitely need to be on Android. And also on the Google Play side, we have a huge amount of activity with over 1 billion people visiting our store every 30 days and interacting with it and play is really helping developers build businesses, really robust businesses that allow you to take global reach and turn them into actually a successful business, whether that means user engagement, revenue or something else for your business. And one of the other things that I'm really excited about here is we've seen a huge amount of this activity happening in emerging markets. We actually see a huge amount of growth on consumer spend and also a majority of downloads coming from this market. So if you're looking to grow, you know, a lot of people start by building products for the people that they know. But as you're looking to expand beyond that, you know, learning to understand these audiences, it's not like emerging markets is going to happen later. It's already happening. And as we've seen from a lot of companies that are actively trying to grow in those regions, there's a huge amount of opportunity there. So one thing we hear a lot about from product managers on Google Play is, well, I don't just want people to use my products, I want them to be able to spend. And so we've really taken that to heart and we have a big team that spends a lot of time trying to bring on new ways to pay. You know, one of the challenges when you have a global market is things like credit cards. People don't necessarily have them in every country. They're very popular here. In places like Germany, people don't really like them. And in places like India, people have still yet other ways to pay. So we've spent a lot of time bringing on these new forms of payment. Now I want to talk about like how to make a successful app because a lot of people talk to me and they say, ellie, I want my app to be really successful on Android. What should I do? And it really starts with building a quality app. But there are some things about the Android platform that can make that a little bit more complicated. So I want to walk through both what some of those challenges are, but also what some of the solutions look like. So we've seen with Android, people really value high quality apps. And we recently took some of Google's machine learning technologies and did some analysis across reviews on the Play Store and in reviews where we could extract and understand the topic of the review, we found out that finished 50% of one star reviews mention some sort of app stability issue like a crash or a bug. So if you want to prevent 1 star reviews making sure that you've thoroughly tested your app and you've used all the best practices and you're preventing those crashes before you roll out to everybody is one of the most important things you can do. And then on the positive side, we found that users respond really well. So in 65% of the five star reviews, users were actually mentioning something like a great design, how fast the app was or how easy to use it was. So all that and design work that you put in really pays off and it will reward you with a higher rating there. And we wanted to give you better tools to be able to interact with user feedback. So we mentioned we were using Google's machine learning algorithms to look at your reviews. And so if you have your app on Google Play, you can actually go into the Google Play developer console and you can look at this type of report. What we've done is basically analyzed all of your reviews. We pull out common topics that users mention about your reviews, and we can even show you how those topics are related to your star rating. So, for example, in this example here, any mention of Klingon, people didn't really like that. So if this was your app, then you can know that people who don't like Klingon are pretty unhappy with your app. And if this was a real app, it would probably make a little bit more sense. So this can be a great way, if you're a product manager, to zoom in on all your user feedback and to understand here are the things that are contributing to happy users and here are the things that are contributing to unhappy users. And it really just short circuits a lot of the process. We used to see where people would have to read every single review and then try to put them in some sort of spreadsheet. And it was a big pain. So we can do better. A lot of folks who are PMs working on Android apps tell me that they are not used to having to work with so many different screen sizes on Android or OS versions. And one of the nice things about Android is that this has kind of been built in from the beginning. Other platforms maybe used to have only one screen size, and it's had to grow and those tools have had to adapt. But since the beginning, we've basically come up with different ways of making it easy to look at that from a design perspective or testing perspective and other things. So if you didn't know, we actually have some dashboards that you can look at on the Android developer website to understand what the rollout patterns look like for different versions and different screen sizes. But one thing that's really important to keep in Mind here is for our most developed markets, your most engaged users, your richest users, they tend to have the newer phones, the bigger screens, all of those things. So what you're largely seeing in these graphs is for the long tail, for maybe if you're looking at a super reach business that's looking at all sorts of emerging markets, they tend to have maybe a bit older devices and things like that. So it's just good for you to know you can also get stats about your particular app. But if you're looking to expand into new markets, this is a great way to understand how you need to make the compatibility work. And so let's talk a little bit more about those Android design tools. So, on the Android Studio side, we've crafted a bunch of tools that make it easy for you to see what your app would look like on multiple different screen types. You can preview it, you can even build Android wear apps in there. Our layouts are built on something called Fragments, which means it's pretty easy to create a layout that will adapt. And you can have different views based on whether someone's building a phone or a tablet view. And then we also give you a bunch of metrics on our design website about understanding what the different specs look like for different popular Android devices. So these can all be great resources to take back to your design team as well. And then finally, we have a lot of support libraries. So just because we have a lot of versions out there in the field, a lot of times you don't have to worry about that as a developer because you can just use these support libraries and with that things will just work great from the beginning. So I want to ask all of you, does anyone here who works on an Android app have a big drawer that they open up and there's like a bunch of phones in it? Yeah. Does anyone have a closet? Well, I have a whole data center now because we realized that a bunch of people were spending a bunch of money trying to source all of these different phones. And that kind of works if you're working on maybe phones that are popular right now in your local market. But. But what if you're trying to expand into another country? What if there's a phone that's popular, that's no longer available, or you can't buy it here in the US So we actually launched a service called the Firebase Test Lab for Android. And what it is is basically a data center with a bunch of phones in it. And Google maintains that. So all you have to do as a developer is Write some tests, and you can run it on our test lab. You don't have to buy the phone, you don't have to maintain it. You can just run those tests. And if you want to get started for free, and if you don't even want to have to write any testing, we actually have a preview of this available in Google Play as the prelaunch report. So the idea here is if you upload your Android app as an alpha or beta version on Google Play, we actually run it against a selection of phones, and you can get a whole bunch of reports back. One of them is crashes. So you can see whether your app crashes on a variety of different devices, and then you can even see what it looks like on different devices. So we'll give you back a bunch of screenshots showing what it looks like on different Android versions, different screen sizes, and even different languages. So, again, if you're trying to build a global business, we're trying to make it as easy as possible for you to have a great experience, even if you're serving users who aren't exactly just like you. So the next theme, this is a thing that people tell me they like the most about Google Play, even though the reason they came to Google Play was to get the reach at Google. We got really famous a long time ago because we weren't really sure what color of green was going to convert well on search links. And. And so we said, well, we're not really good designers, so we'll just run an AB test. And we've also done a lot around beta testing. So Gmail was in beta for a long time, for example. And what we really want to do is to make all of these great tools available to apps, because, let's face it, apps can be kind of static, right? We had the web where you could update it continuously. And then with native apps, we're moving to a world where a user kind of has these concrete updates that need to be applied. And so when you want to push something out there, you want to make sure that you've really polished it beforehand. So earlier we heard from some other product managers talking about things that happen when you don't quite have the right product market fit, right? And we've all been there. None of us want to be there again where we've launched a product. And then later we realized that it wasn't really serving a real user need. And so I try to force myself to watch Silicon Valley sometimes, but it's really hard for me because it's way too close to home. But they had some recent episodes that were about the perils of beta testing gone wrong. Basically, they had done a beta test of their product with a couple of their friends and their friends were like too nice to kind of say that it wasn't that good. And maybe some of their friends thought it was fine, but their friends weren't really the target users. And so as a result, they had a big failure. Right. And we all talked about how we want to fail earlier in the process. We heard some great stories earlier about testing prototypes and mockups rather than building the whole thing. So on Google Play, we've tried to put in place a lot of tools to make this easier. So if you talk about launching an app, you want to talk about not just launching it, but what happens before that? Full launch, beta testing, doing a soft launch that's particularly popular in the games industry. And then you can get to full launch and then eventually, you know, rolling it out. And so when you think about this, this doesn't just apply to new apps and games, but also just any new feature. So let's take a look at what soft launch could look like. So initially, you know, maybe you have a very closed group. You initially just want to get feedback on your overall concept. Maybe you want to restrict that to people who are willing to keep it confidential. Maybe you're just looking for bugs. And then later maybe you open it up a little bit more and you're looking for more product market fit, data. Maybe you start to collect metrics. And as you open it up more and more, you have a less and less biased sample. Right. Initially, if it's just friends and family or employees, they're not going to be very representative. Then you're going to move to earlier adopters that are more representative. And then finally you'll be out in more of the broad population. And so you'll have different goals at each different stage here. And so here's some of the tools that Play has developed to try to make this easier. So about three years ago, we launched a beta testing product. Has anyone here used it, by the way? Has anyone run a beta test on Android? Handful of people, actually. If you look at our top apps, over 60% of them have tested, have tried this and are actually running a beta right now. They like it so much that they keep on doing it. It's actually a best practice at Google that will do this in different ways. And this is a really nice way to be able to just put your app out there and we manage it for you. You don't have to install any separate software. People can just download the beta from the Play Store and it gets automatically updated. And here's how it works. You can either make a closed beta where you control who can access it. So maybe that's when you're in stealth mode and you invite a set of people. People. Then you can also open it up. Right. We have other very big customers who use the open beta product to get feedback from literally millions of people. People can opt in right from the page on the Play Store. You can run two different tests at the same time and then you can get private feedback. Right. That beta feedback doesn't show up as a Play Store review. And we got a lot of good feedback on that. People said that was really cool, but we were talking to some of our folks in the startup community here and they said, well, I'd love to do a beta test beyond my friends and family, but we're a startup. Who are we supposed to ask? We don't know anyone to ask. And what we realized on Google Play, like we said, we have about a billion people going there every month. And so here we had a bunch of startups that really wanted beta testers and then we have a billion people who are looking for cool stuff to try. And so we created something called the Early Access Collection. This is a special place in Google Play where we curate a set of the most interesting new pre release apps and games. And so those folks can get early stage feedback and people who are early adopters and like to try new stuff can have a place to go. So this isn't necessarily representative of a mainstream audience, but it definitely is a great place where if you have a new product that you're really excited about, you can get much more broad feedback very easily and cheaply. And then the next product we have around iteration, we launched last year. It's called Store Listing Experiments. Has anyone here tried that one? Okay, well this is definitely one that you'll want to think about. We have over 70% of our top developers using this one. I'll describe a meeting that has probably happened at your company. Let's say you're working on launching a new app or game. There's three different versions of the logo that the designer is proposing and a couple people from the team get in a fight. I like this one better. That color's ugly. I think we're better represented by a turtle. No, that turtle is terrible. We should use a cow instead of for our logo. Right. This meeting could go on for a long time. Right. And who knows? When you create your listing on Play, you really want to make it pop the first time. So rather than having to have an argument about what icons to use, which marketing copy is going to be more effective, we let you run that as an a B test. You can actually run up to three different variants. You can get a report that shows what the conversion rate is going to be. And then when we get enough data, we say, hey, your experiment is completed, it's statistically significant. And then you're able to apply the most effective version. And again, this can really streamline a lot of discussions that you've had. And let's talk about what some of the results look like. We found that developers who start using this just keep on using it again and again because they get pretty significant increases. You don't have to write any code, you just get a copywriter and a designer, try a few different variants. Here are some examples of some developers who tried it. And we find that when they do experiments again and again and again, they get pretty big gains. We find the icon, by the way, has the biggest impact. So if you haven't tried a store listing yet, you want to try your experiment by starting with the icon. That's definitely a good place to go. And here's an example of Monster Strike. Does anyone know Monster Strike by Mxy? It's really, really popular game, especially in Asia. So what they did is they were trying to bring their app to the US and they didn't know what kind of graphics and text were going to resonate with users here. So they did a very, very methodical approach of looking at every single aspect of their Google Play store listing, running several experiments and optimizing it. And so at the end of the day, through a series of experiments, they were able to increase installs by 68%. And this is for a very large app. And so if you're doing any sort of marketing for yourself, right, if you're paying for ads, if you're even trying to have a viral, what you definitely want to do is to make sure that the people are not dropping off. And when people go to your Play Store page and are making that decision about, okay, I saw that ad, or my friend told me, do I really want to install? You want to reduce that drop off as much as possible. And store listing experiments is the best way to do that. It'll make all of your marketing more effective. And then here's another example from Rovio of what happens when you run those successive experiments. You take a hypothesis Each time, test it, pick the best variant and try again. And so again, they were able to make a significant amount of increase in their install rate when they launched Angry Birds, Transformers. And then they took these findings and they applied them to their marketing across all platforms, not just Google Play. And finally, I wanted to talk about Instant Apps. Has anyone here heard of Instant Apps? We announced them at Google I O this year. I'm very happy to have some members of the Instant Apps team here with me today. And this is the one feature that I'm the most excited about in Android right now. We talked a lot about these usage patterns with apps and there will be some apps who are those best friends and that everyone will want to install them and use them 85 times a day. But what if you're the sort of app where you only want to be used sometimes? And as a product manager, you want to bring someone into your app as quickly as possible, right. You don't want them to have to wait through a 50 megabyte install. So how Instant Apps works is if someone just sends you a link, maybe it's through a message, you can just click on the link and go directly into the app. You don't have to go to an app store, you don't have to wait through a download. It's just click and then you're in that experience. And it's really great. You can unlock discovery through search really easily. You can accept digital payments, you can just go directly into that app experience. Right. If we think about what was great about the web, it was that was so easily accessible. And Instant Apps is really closing that gap there. So if you are interested in being part of this. Instant Apps is not really out yet. It's not available. But we are slowly planning to start bringing it to more and more people. So we'd love to hear from you. We have a link to sign up to hear more. I'm really thinking it's going to do a lot. We shared a couple of examples at Google I O earlier this year. We showed how with buzzfeed Video, you can share links to really interactive app experiences. We showed how through shopping with B and H photo and video, you can go from a Google search directly into an app again without an app install. And finally, how Instant Apps could interact with the physical world. So if I had my phone, I could just walk up to a parking meter, tap on it to pay, and without ever having that app installed, I could go into a full integrated app experience with all my payment information. So that's it. I talked about four different things today that I think every product manager should know about Android. First, the reach to a global audience is really the number one reason people come to Android. Number two, building an app that works on the huge range of Android phones and all the diversity there can be a bit intimidating, but we actually have a lot of tools that can make that a lot easier. Number three, people tell me that once they come to Android and Google Play, this is the number one reason they love the platform. And then finally, instant apps. It's a brand new technology that we're going to be rolling out soon and we're excited to be able to bring people into app experiences faster. Any questions? That's a great question. So with instant apps, you can actually get directly into a native app experience. And so for certain types of apps, that can be really important. Just the way that you can interact with them, like scrolling very smoothly through results, the richness of the experience, playback. It's one of those things where it's really hard to describe, but once you try it, it's really, I don't know, it's that je ne sais quoi. Like if you go to a website for something and the app for something, people with the two side to side tend to appreciate the app a little bit more. We like to say it's sort of smoother and richer.

How much more coding is required to have.

Yeah, so there is a bit to get started. It's really about more like taking your existing Android app and then upgrading it, kind of breaking it apart into different modules. Once you've done that, it's more maintainable. So it's not like you're creating a separate app. It's more like something that you do to your existing app. But there definitely is some work to do to unlock that. Any other questions? Yes, Instant apps. Sorry, your question was whether instant apps work on iOS? So instant apps is exclusive to Android. So if the idea is that it works with a regular URL. So on Android, if you click on a URL and that is enabled for instant apps, it would just work and it would bring you into the instant app experience. If someone sent you maybe through messenger or something like that, a link on iOS, it would handle that either through the native app or the web, depending on how it was set up on that phone. Any other questions?

What do you think about the role of games inside of your ecosystem? I've noticed there's a lot of tension between apps and games, like on both sides of the equation. So curious about how you think about it.

Yeah. So the question was about what is the role of games on the platform? They're really important. I think one of the things that we see a lot of difference with is like, here in the US Games is a thing that, you know, maybe some people will do. You know, Candy Crush was popular with a lot of folks, but if you look at other markets like Korea, Japan, Taiwan, it's something that everybody does, you know, like maybe think of the amount of time people maybe spend in social and messaging experiences. A lot of more people are spending in game experience experiences there. So especially with Android, we have a really, really strong presence in Asia. And so if you look at kind of the amount of time spent on games on the Android platform, it's really huge. And I think for a lot of people, when they're choosing a device, if games are important to them, it is something that they think a lot about. Yes.

So I know Apple kind of takes a pretty editorial approach towards curating their app store. So how do you think about balancing that versus more programmatic?

So the question was about programmatic versus curated experiences. I think that's a really good question. On the Google Play side, we actually take both things very seriously. I'll give you an example, right? Like if when Pokemon Go launched, the data would not have shown that Pokemon Go was the app to recommend that day, right? Because there was literally zero data. But if you went to Google Play and Pokemon Go was not the first thing you saw, you would say, all of my friends are talking about this is all over news. It should be front and center. And then also when we're talking about, let's say, growing startup communities, right? You want to be able to say, here are a curated set of things that we think are very important. Even again, if the data wouldn't show that. So in the Google Play team, we look at both areas, we look at both people, people who are experts in each of the different areas to try to understand what's new, what's coming out, what's really interesting and unique, as well as obviously, Google's very famous for our machine learning and other capabilities to look at data and personalize to the particular user. And I think also when you have over 1.4 billion users active on Android, this becomes more and more important. Right. Curation works to an extent, but for that many people, you definitely need to have a big automated aspect as well. Yes.

Quick question about, for the iteration tools, which is great. Someone like Angry Birds, they're getting a lot of people to go to quickly learn which Ones are working? Which ones are not working? How do you handle that? They're like, hey, I'm Johnny Developer and I've got an app I'm trying to get. Like, figure out what is the best way to market it.

Yeah, that's a great question. So the question was, if I have a lot of traffic coming to my Google Play Store listing, I can run a lot of experiments very quickly. If I don't have a lot of traffic and I also want to know what graphic is best or what text is best, what do I do? This is a huge challenge, actually. We've had a lot of that feedback from startups. I think early access is one of the places that we wanted to do that because, you know, again, if you don't have enough data, you can't do a statistically significant experiment. You know, people will try different ways where they'll try to, like, how do I put this? If you are planning a marketing campaign, maybe you do a little mini one in advance to try to generate data to be able to run that experiment early. And then maybe when the holiday's coming up and your big campaign's going to run, you've learned. So people will sometimes try to time their experiments with either paid or organic marketing efforts. It is hard. And I think that's one of the big reasons we brought out Early Access was this was an ongoing piece of feedback from the startup community, like, I don't have any traffic. Can you guys help me with that? With your billion users? And we said we should probably do that. Okay, other questions. One more.

Can you talk about the interaction between AMP and instant apps? I know they're coming from Google. They've stated that they're going to load an AMP to the AMP page if you have it, which means they wouldn't load an instant app. So as someone building that experience, should you load an AMP page and then think of anything else? Try to instant app?

Yeah. So the question was about amp, which is a web page acceleration technology from Google versus instant apps. There's a lot of things that we're trying at Google to try to be able to bring users into experiences as fast as possible. And one of the interesting things about being at a big company that works in a lot of different things is there's different potential solutions and we don't know yet which one is going to work the best. You know, instant apps isn't even out yet. So what we definitely are recommending to developers now is if you've invested a lot in building a great website and great web based content amp is a great way to accelerate that. If you've invested a lot in a native Android app and you want to accelerate that, then instant apps would be the way to do it. And if you have both, it's a little bit more tricky to figure out. We're also still learning. You know, Instant Apps isn't out yet, but it would be up to kind of the handler on the device itself from a technical perspective to decide what experience would get opened. I think in general, if it's trying to handle it as a native application, we would go to the instant apps experience, because again, that's the richest experience. That's native code running on the device with the full Android device capabilities. Cool.

Thank you so much, Elliot.