Managing Product Operations at Scale w/ Matt Shroder, Former Head of Product Operations at Uber

Hey everybody, this is Mike Belsito, a co-founder of Product Collective, and we're back for another industry interview. Happy December to everybody. I know we're now definitely thrust into the holiday season here, so I hope it's going well for all of you. I'm excited for today's session. We actually have a guest who helped out with a resource that we recently launched in collaboration with Pendo. Which was The Rise of Product Ops. It was an ebook that we put together. But for that ebook, we went through all these interviews with product operations leaders, and Matt Schroeder was one of those leaders. And so I'm excited to have Matt with us to talk about his experiences. He most recently was on the product operations team at Uber as a leader on that product operations team. He recently left that team, but is one of the people to sort of— one of the go-to people in the product ops world. And was highly recommended to us. What I do want to make sure is that if all of you are tuning in for the very first time, you know kind of what the lay of the land is here. So I'm going to share just a little bit about that first. Um, some of you are already chatting, which is great. That chat, actually, it's for you to introduce yourself, share where you're from, uh, maybe what product you're working on. It's always fun to, to see where people are tuning in from. And actually, just for my own sanity, if you can see and hear me okay, you Maybe just put that in the chat, so then it'll make me feel better that I'm not talking to the abyss here. But these chats are with product leaders from really all over, and we cover a different topic every single time. And again, today we'll be focused on product operations, but you can always check back for future editions. We'll—

oh, you can see and hear me. That's great.

Thank you for that. But you could check back for future editions. We've been doing this on Crowdcast. We're checking out some other platforms too. Keep an eye out for more information on the, on additions that we have in 2020. If you're not familiar with Product Collective, maybe just stumbled upon this video chat, this is one of the initiatives that we have here at Product Collective. Product Collective is a community for product people. Aside from these industry interviews, which we do twice per month, we have a weekly newsletter that goes out to about 25,000 people. We have a pretty active Slack community. I think there's about 10,000 people within the Slack group. And then we all meet in person at Industry. Industry, the product conference, is the conference that we organize. And actually there's two of them. We have a European edition in Dublin, Ireland. That one's coming up in March. And then we have our global edition, which is every fall in Cleveland, Ohio. And actually today we just announced, uh, we have a— in collaboration with Balsamiq, there's a scholarship program for the European edition of Industry. We're giving away 25 full conference passes as a part of that scholarship program. So you could learn more details at europe.industryconference.com, and there you'll see a little scholarship. In the meantime though, I want to get right into the discussion today. Just before I do, I mentioned the chat. If you have questions for our guests, you can ask questions really at any time, but there's an ask a question section. I'll be monitoring that throughout, and maybe it's right in the moment, maybe it's towards the end. Whatever it is, I'm going to be pulling questions from there. I have a lot of questions for our guests, but I'd much rather ask your questions because they're probably better than my questions, to be honest with you. So at any time you— that something pops up, load it in that ask a question section and I'll keep checking back there. But with all of that, I want to introduce you to our guest who is actually an Ohio native like me, and that's Matt Schroeder, again, formerly one of the product operations leaders at Uber. Matt, it's awesome to have you with us.

Thanks for having me. Hope everyone had a great Thanksgiving holiday.

Awesome. Well, you know, I kind of want to get right into it. And, you know, you and I, we were talking about this a little bit before we went live, but, you know, we first got in touch when we and Pendo were working on this Rise of Product Operations ebook together. And part of the reason why we collaborated on that ebook is that product operations is this— it's sort of like this thing within the product world that is super important for some companies it's downright critical, but there's still not, it's not necessarily, I don't wanna say widely known, but I would say maybe there's still a lot of questions in terms of like, what exactly is it? Does my company need it? How do we get into it? So maybe if it's cool with you, like I'd love to just get your take on what is product operations in the first place? Maybe how would you define it? And then we could kind of take it from there.

Yeah, I think it's a great point. You know, when I started with the product operations team at Uber, I previously was part of Uber's central operations team based out of Chicago, and we needed a centralized operations team to help with our product strategy. And so it was asked 4 years ago, but when we started that team, product operations wasn't really a role anywhere around the industry. And it's been interesting to see over the last 4 years, similar to what our research has shown us, there's been a lot more companies who stood this thing up. I don't think the roles and responsibilities, I think it's important to say, aren't consistent yet, but I think over time we'll start to see at least some more consistencies in what these folks are doing. So how we define product operations at Uber, it's operationalizing our business strategy through the product organization is, I think, the most simple way for me to say it. And I think why we invested in product ops, it's really important that you know about Uber and the strategy of Uber But we are a digital meets physical company. And so anytime that we build product, those things are facilitating real transactions or real humans on the ground. And so building products, especially in a global context, is super challenging for a central product team. And so we had to create one central operations team that looks across multiple business lines and is able to say, you know what, this is the right priority. These are the right things to think about when we're building you know, platforms or products to enable our operations team to compete in a hyper-local situation. So to kind of summarize, what are the areas that Uber Product Ops today own? I think, one, it's important to note that what we originally stood up for is not what we're doing today, which I can get to in a second. But our three areas of ownership today are, one, the company-wide planning process. So who's responsible saying, you know, these are the strategies of the business, this is what product needs to invest in, and how do these things intertwine for one, you know, strategy for the year and product roadmap? Two is actually— two and three is actually now when you think about the product development lifecycle from the very beginning, when you're thinking about, you know, sort of insight generation and problem, you know, overviews. We are very much helping compile what those insights are across all the geographies and the consumers that we build for. And then the last thing that we own is sort of the go-to-market execution. And there's a lot that goes there from, you know, testing the products, making sure that the localization is correct for that particular country or consumer, to, you know, basically building feedback loops with our support organizations so that As we launch these products, very much our product team and our tech teams understand what the impact is on the ground and can quickly make iterations if necessary.

Awesome. I think that's a great starting place for us. What I'm curious about is, 'cause you were at Uber at a time where it experienced tremendous growth. And I imagine that product operations as a role Like there was some point where it became critical for Uber, but I'm just thinking even outside of Uber, certain companies, there's probably a time where it becomes critical for a certain reason. I, I'm just curious, what, like, when is that point? Like when, when does product operations as a function become so important? Um, or are there some companies where even right from the get-go it, it's something that is that critical? Maybe you could talk about that a little bit.

Yeah, I think it probably changes company to company, but I can give you a little sense of what we were going through. So at Uber, we had a very decentralized operations team, meaning every city or every country was basically treated as its own company, meaning they were responsible for the P&L ownership and things like that. And so, you know, over time, especially when you think about Uber circa 2015, suddenly we found ourselves in 120 cities that were all kind of operating, you know, by themselves. And I think one, that was a good thing for Uber at the very beginning because being first to market and actually having an in-market presence is so incredibly important for an industry like ours. And so I think it was the reason behind a lot of our growth. However, when you become so decentralized by that, it's easy to find those activities that you could easily centralize. And so product operations actually first started to say, all right, this process is being handled 120 different ways. How do we centralize that activity and have, you know, more efficient or fewer resources doing that same activity? And when you can standardize that process, then it's right for automation. And so then it became, you know, this operation team that owned these central processes, now inserting them into the product, you know, planning process to be automated. Now, over time, the needs of the business have changed. So when you talk about, you know, Uber's geographical expansion, especially in 2015 to 2016, you know, we were at first just like a US-only market, but suddenly we found ourselves in 40, 40+ countries. And when you think about at least how our industry operates, you know, there is a hyper-local competitor in, in almost every single country in which we exist. And so it's a really tough thing to think about, okay, well, how do we build a global strategy and utilize the most of our resources so that what we build actually does scale globally, but allow it to be configured in a way that can compete with a hyper-local competition. And when you think about the challenges there, it definitely changes your product strategy, it changes how you operate internally. And that was really starting to pivot the product operations team to think about, okay, well, what's the insight gathering process? Is how do we get ahead of, you know, regulations or archetypes of users because they are, you know, so different. If you think about just on the rider side, you know, try building an app in the US and compare that to, you know, the average consumer that you might see in, you know, Egypt or India with, you know, low bandwidth devices, not as strong infrastructure. It is a much more challenging thing to do from a global perspective.

Totally. I, and, and I appreciate you, uh, sharing that piece of it. I, I am curious, is there a, like, what is the difference between somebody that might be just a product person? Because at Uber also you had roles before it was defined as product operations, like there were roles specifically in product Like, I'm curious, what's the difference between that product manager maybe at a certain company and then getting into product operations? Like, does it take— what type of product manager does it take to sort of excel in a product operations-specific role? Or, you know, are there characteristics that maybe even as a hiring manager that you're sort of looking for? If you could talk about that.

Yeah. So I think it's a great point. I don't think they necessarily are the same. Skill sets initially. I think over time, you know, when you, when you work as close as we do with the product management organization, then you start to see, you know, you're sort of changing career paths and things like that. But when we first, you know, try to hire folks onto our team, there's certainly more competencies that we look for. So, you know, if you think about, you know, this is basically a Biz Ops team for product, right? And so what are the qualities that you would look for in someone like that? So one, very strong business acumen or even preferably like a strategy background. So what are those products that we need to build? How do we get ahead of regulatory changes? Can we foresee that? The second thing is a solid tech foundation. I don't necessarily think— and what we look for is you don't need someone with product experience, but you need someone with the basic understanding of how it works. What's mobile build deployment look like? If someone within our business says, hey, we need help to prioritize this solution to unlock this market or whatever it might be, do you have a foundational understanding of that? Is it a big ask? Is it a small ask? How long will it take? Those types of things are things that you still need to have a very strong understanding, but you don't necessarily need to be an engineering background like I see in a lot of product managers. Much more on the business acumen, you know, MBA consulting type backgrounds is what we traditionally have seen in the past.

Sorry, I had to unmute myself as I'm searching for the unmute button.

Sure.

I had an audience question come in and actually this is a great reminder to everybody. If you have questions for Matt, bring them in, submit them in the ask a question section because I'd love to be able to get to as many as your questions as possible. And this pretty unique opportunity to have somebody like Matt available to you to field your questions. So this question is specifically about Uber, but also, you know, if you want to kind of answer it more broadly, that'd be fine too. But the question was whether there were multiple roadmaps for Uber, or by owning product planning, does the operations team actually own that global roadmap, which would include global features, but maybe also individual country-specific features. So curious, uh, your take on that.

Yeah, uh, so let me pop into the question. So the question was, what was the first part of that? So there are multiple—

like, does the operations team own that global roadmap and, and they're, they're sort of managing that? Or like, how does it work with global features versus those individual country features? I remember you saying before how, um, some of those individual countries or even cities sort of operate independently.

Yeah, I wanna— I think it's important to clarify that what we're summarizing or what we're owning in the planning process is more saying, you know, these, these are the problems that need to be solved for in sort of a holistic context for the business. So being much more specific about what are the insights and the pain points that exist from our business, can we summarize them from a geographical perspective and say, you know, 50% of our business thinks this is the largest issue. We try not to get too prescriptive in product solutions. Because we know that our product manager counterparts and especially the design team, you know, if they have a really strong understanding of what the global insights and problems are that exist, then they are probably more likely to build better solutions than what we could ever propose. So I want to kind of clarify that. But related to Uber, you know, do we have multiple product roadmaps? So I think there's certainly with different business units, they have, you know, different roadmaps. So for an example, an Eats isn't necessarily always going to tie exactly to an Uber rideshare business. However, we do have more company-wide priorities that theoretically, you know, cascade down to the various organizations into the various teams so that every team, you know, has some layer of KPIs that roll up to a strategic priority. And I think having those cascade cascading KPIs and OKRs, and especially having the accountability mechanisms, whether that's business reviews or quarterly updates, are super important for how we make sure that, you know, we're accountable for delivering against those results. But, you know, one thing that I would say about Uber, I'll conflate this more to the rideshare business, but, you know, we operate in a very heavy regulated environment. And if you just think about the nature of our business for a second, so, you know, every city has different transportation or different transportation or municipality laws. And so that could be something like, you know, this driver needs a particular license to New York, or this vehicle has to be, you know, licensed. I've seen some cities have, you know, weird regulations such as, I believe it's Cairo, you have to have a blood test done And these are all requirements that Uber has to build within its process. So that, that's one, right? Municipality and transportation laws at a city level. And then because we also operate sort of in the independent contractor area, we often have to pay close attention to federal employment laws. And the reality is, is if you kind of stack rank those, if you're a product manager who's sitting in San Francisco, you suddenly find yourself trying to solve or build a product solution for this Rubik's Cube of regulatory requirements that's constantly changing. And so when you talk about where they very much rely on product ops and saying, okay, well, where do those problems exist? You know, if we're trying to get into a new business line, can we scope what those are? Can we summarize them into, you know, either market regulatory archetypes or consumer archetypes and then kind of deliver this, Well, if we can solve for these 4 problems, then we know that we can solve for 80% of the things that exist in this particular space. Then if we want to invest in that additional 20%, if the market makes sense, then we can go invest in that. So hopefully that gives you a better sense of like why this is so challenging for someone, you know, who's essentially building things in San Francisco for a global business. And what that actually translates to is a lot more of, you know, if you think about how to compete in a hyperlocal market, It's a lot of platforms. How do you enable your, you know, your teams on the ground to, you know, configure something on a backend as opposed to, you know, constantly coming back to the headquarters team and potentially, you know, throwing a wrench in that roadmap? And that's one of the things that we ultimately want to avoid is, you know, we want to let our engineers and our builders build. And so getting ahead of those things are super important to us.

That's great. I wanted to go back. I'm seeing questions coming. Sure. That's awesome. Well, one question I did want to ask that I had on my end was just what is it like? And again, I know it'll differ or vary company to company, but what is it like in terms of working with the rest of the organization? As I imagine, you know, having so much data and that data is important, all sorts of different groups in the company, you know, sort of view that as mission critical. So How does that role interact with other areas of the organization?

Yeah, so I think, you know, certainly the two functions that we interact with the most is certainly one, the operations team. So you have to have very close ties to, you know, the folks on the ground, the folks who are actually truly being the face to our customers, because the further you get away from those customers, the less likely you're going to be building something that's really customer-centric. So, you know, constantly, we're constantly in region visiting them, I'm trying to understand the pain points that exist. The other one, the other function that we very much spend a lot of time with is the product management team. So when, especially when you talk about sort of the insight generation process, making sure that you're there for, you know, overview of the problems that exist, working with them on the go-to-market strategy, if, like, you know, in the middle of the testing or localization process, you realize there is a string that is completely, you know, broken to a certain set of consumers, or something just isn't right with the product. How do you work with them in sort of the sprint planning to kind of get that fixed? These are all things that, you know, we have a lot of overlap with those two teams.

And this might be a good lead-in to this question. This is an audience question, and it was How does product operations fit into Agile Scrum teams? You know, so sort of for people that are familiar with, with that methodology, like where does this sort of plug in?

Yeah, that's a great question. So we do sit in, at least this I'm speaking from my perspective and my team's perspective, we do sit in sort of the weekly sprint planning meetings. So specifically, where we probably bring the most value is sort of bringing insights from our support organization or from the go-to-market strategy. So for example, if we know that there is a bug specifically that is causing a lot of chaos within our support organization, or we know that through sort of the testing that the folks are doing on the ground ahead of these product launches, if we can bring those problems back to the Scrum planning meeting and say, from an operations standpoint, these are the 4 or 5 things that are really holding us back, that is where we sort of introduce, you know, our insights and pain points into that process. But for the most part, you know, outside of those prioritization, you know, sprint planning meetings, you know, probably less involved than a product manager typically is. I also see a question from Michelle around research ops and design ops differing from product ops. So I think it's a great question. The way that we very much see that is, you know, when we talk about UX research teams, our teams very much are more so focused on, you know, product operations will bring the problems and say, these are the things that need to be solved related to our business priorities, right? So maybe we need to reduce costs or maybe we need to focus more on growth and profit. Than profitability or vice versa? Who's responsible for actually sharing that strategy or summarizing it? You know, what are the issues that exist within the regions? Whereas a research team is more so focused on, hey, we have these multiple solutions that we've come up with the design team. Who's actually sitting down with the customers of, you know, particular countries that are super valuable to us, the different consumer types that we work with, and really understanding, you know, do they, do they Is the UX right? Do they understand what we're trying to do here? Is the product that we're building helping to solve for the solutions that were originally outlined in sort of the summaries that Product Ops helped create? And those are the things that a lot of times we collaborate with the UX research team on, but they are very much focused on more, you know, what types of iterations of the design do we need to continue to work on? Is this solving what we need to solve for, etc.?

There is another audience question that came in, which has to do with sort of building up product ops within an organization. And their question specifically was about, hey, if I, you know, I'm in product at this company, I think this is where a company should go, but leadership might not even be thinking about this quite yet. Like, what can I do to sort of best sort of— I don't want to say sell this up, but You know, it's like, what can I do to help my organization maybe better understand the importance of product operations?

That's a great question. I think, you know, I think we had the luxury of when we started the team, it actually came top down. It was very much, you know, our head of operations at the time saw the pain that the operations team was having because I guess the operations team probably didn't feel like the tech strategy or the products being built were, you know, global in nature and they were much more specific to one particular country. And when I see that happen, generally what I, you know, see in other companies that I look at this into for is there's not one central team looking across all the different businesses, if that makes sense. Meaning, you know, you have your central product team, you have your different, you know, geographies or business units or whatever you want to call it, but then there isn't, you know, probably smaller in headcount, one team that sits across the business and saying, you know, if two folks from different business lines go to a product manager about something, how is that product manager supposed to understand, you know, what truly to prioritize in the needs of the business? And so I have seen a lot of pain points exist from that. And I would say to the user who asked the question, you know, if that, if that's something that clearly is missing, I think they're generally— what I would do is I generally get feedback from my product managers and engineering counterparts and say, Do you feel like you have a clear one-to-one relationship within the business, or do you feel like suddenly you're getting pinged from a bunch of people in a bunch of different verticals? It's not necessarily going through true prioritization meetings or whatever it might be. And how do you summarize that? And the best way that I've done it in the past is, I've created the internal surveys that I'm sure so many people hate, but Really, my point was, do you feel like you have a clear connection to the business and do you understand the business strategy? And if the product team comes back and says, you know, no, I think that's a huge concern. And I think that you're able to take, you know, those results or whichever mechanism you want to use to, you know, record that and, you know, just have a dialogue and say either the organizational structure is not right or we're not giving the clear, you know, point of contact in the business. Or we need this central function to really help sort of coordinate the two organizations across.

Thank you for that. And actually, one more audience question coming in. I know we're getting close to the end of our time, but I definitely wanted to get this one in, which is, how do you, as a Product Ops team, how do you measure success? Like, what KPIs does a Product Ops team live with? Is that something that's just sort of dependent on your sort of corporate KPIs, maybe you could touch base on that a little bit.

Yeah, so I think, you know, depending on your area, you're gonna definitely have different KPIs. So for example, if you're like supporting a platform or a marketplace, you know, that's gonna be looking quite a bit different than if you're consumer-focused on like a rider or driver. However, there are a few things that are pretty consistent. So one is we do have some layer of consumer satisfaction score for internal stakeholders. So we have consistent surveys that we send on a quarterly basis to not only the folks on the ground, but also the tech folks saying, do you feel like you're aligned to the issues that exist in the business? Do you feel like we're operating at a high level against our strategy? Those are types of things that we certainly survey for internal. But then if you think about the actual KPIs of what the role is doing, you know, there's certainly in the insight generation process, there's certainly more of a, you know, qualitative perspective. And that's where we really rely on peer reviews and being able to identify, you know, are these truly the right things for the business? Are they summarized in a way that, you know, the data speaks for itself, but also the go-to-market execution. We do hold our product operations team members accountable for things like, you know, during launch, are defect rates or support tickets spiking or are they going up? Or are you building solutions that, you know, it's making it easier on the consumer and, you know, hopefully reducing support rates? Time to market. Is this something that, you know, once we test, we've, you know, scoped it enough, we built the right product solutions that suddenly we can launch it in 2 weeks? Or is this a product where very much we built it, we had launched it in one market, and then it takes us a year to roll it out. So those timeframes are also super critical in helping us understand when we summarize these issues and we think about the product solutions, are they actively— are we working with our teams to actively build something that can be scaled quickly? And in our industry, that's super important. And just to give you an example of why that's important, And I think many industries are like this, especially for companies that are very global in nature. You could launch something and test something in one country, and by the time you feel really good about launching that, competition in another country has already launched it to their local consumers. And so being able to quickly test and quickly follow up is so incredibly important.

Yeah. Well, I appreciate this whole chat. I think this is awesome. I'm getting awesome feedback in the chats here. We're pretty much at the end of our time. So I want to thank you for being a part of this. Maybe if we could, you know, if there's sort of one big takeaway that you want to make sure that, you know, people leave with, you know, they tuned in because maybe they're curious about product ops, that this would be a quick one, but just what is that one thing that you want people to know about product ops that maybe somebody that hasn't spent a ton of time in it, like they wouldn't even be thinking about this? What is that one big takeaway for you?

Yeah, I mean, these are certainly emerging teams. I would say that You know, the caveat to this whole conversation is, based on our research, what we have learned is there is still a very wide range of roles and responsibilities. And I do think that not every, you know, company out there needs to have a product operations team. I think it makes much more sense where there are companies with large operations teams that exist, especially ones that don't have those centralized functions that can help sort of operationalize that tech strategy piece of it. But also, you know, in places like digital meets physical companies, this is very much an emerging role that I think over the next 5 years will start to become more consistent and probably, you know, be around for the long haul.

Well, hey, Matt, I appreciate you taking the time for this. So thank you. Thank everybody for tuning in. Appreciate your time as well. As well. And yeah, the recording for this session will be pretty much immediately available after this session. But yeah, stay tuned for future industry interviews. And yeah, thanks again, Matt.

Thanks for having me. All right, take care.

See you later, everybody. Have a great day.