CTO Leadership Lessons with Intuit CTO Marianna Tessel and Andreessen Horowitz GP David Ulevitc

Thank you for, for being with us. So real quick, we're gonna be talking about people, processes, and priorities. We are very lucky to have Mariana Tessel, the CTO at Intuit, with us. I'm going to read a quick bio and background for her and hopefully not embarrass her because she is quite accomplished. We're very lucky to have her with us today. Real quick, before I do that, I'll give you my quick introduction. I'm David Yulovich. I'm a general partner at Andreessen Horowitz. I invest in enterprise software. Which is why I'm very lucky to be able to have this discussion and interview with Mariana. Before that, I ran the security business at Cisco, and before that, I was the founder and CEO of OpenDNS. I would say I'm a technologist, but nothing compared to Mariana. So let me, let me give you quickly her background. Mariana grew up in Israel. She is an adventurer. She's done backpacking trips through India, Nepal, Thailand, She was a captain in the Israeli military, and she came to Silicon Valley after all of that and started General Magic, a famous startup that was way ahead of its time. But she has also gone deep into systems. She was an engineering leader at Docker, where she went deep into containers and security and microservices and the entire DevOps movement. And then she moved over to Intuit, which is where she is today, and we're going to be talking about her career and trajectory through engineering leadership, how she leads teams, how she thinks about scaling organizations, building great products, being a leader in organizations of all different sizes and scale. And in her personal life, she has 4 kids and she juggles all of that while also being CTO of a major corporation. So we're very lucky to have you, Mariana. Thank you for being here. I just want to make sure all the video and audio and everything is working.

Thank you, David, and thanks for embarrassing me with the bio.

Of course. Well, it's great to have you. And just so everyone knows, just a quick few little bullet points here. We will be talking about both really, I think, nuanced and specific topics and also, you know, sort of buzzwordy cliché topics like the future of work, which we are all literally living at this very moment by definition. We're going to be talking about trends in the industry and how things are changing. How the way teams come together, recruiting talent, you know, how do you retain the best people, how do you motivate people, and then obviously we're gonna be talking about leading and scaling engineering orgs. Rather than saving all the Q&A till the very end, we have a Q&A box you can use to submit questions. I'll be monitoring that through the discussion, and so if you have questions, please drive the conversation, that's why we're here. I want to make sure it's valuable to all of you. And then obviously in the intervening time, I have a whole bunch of things that I'm excited to ask Mariana about and to talk about. And so with that, we will jump in, but feel free, you don't have to wait till the end for the Q&A, just use the box. I'll pull the questions and then I'll do my best to answer them. I'm also going to stop the screen share because I think you'd rather see us instead of all the slides. So hopefully this is all working. And with that, what do you say we just jump in, huh, Mariana?

Let's just do it.

All right, so why don't we start with just give us some background about your current— what is your current job? What's the size of your organization? What do you do? What products are you responsible for? And we'll go from there.

You know, like I said, I'm a CTO at Intuit, and we're a mission-driven global financial platform company. Actually, our mission is to power prosperity around the world. So very inspiring for us internally and for our users and customers. You know, we are— probably a lot of people know us as the makers of TurboTax. Hopefully you submitted your taxes on July 15th this year. QuickBooks and Mint. So those are our flagship products. And, you know, my responsibility in the company is to drive technology for our 50 million+ customers and, you know, decide on the technology vision and mobilize the teams to deliver.

Excellent. Well, you don't just get into that job having done nothing before. So why don't we learn a little bit about your career trajectory? I sort of gave the adventurous fast-forward version from General Magic to Intuit, but why don't you talk a little bit more about how you fell into engineering leadership, you're, you know, from technical, you know, I don't know if you were ever an individual contributor, And then a manager. How did all that come together? Walk us through your career.

Yeah, sure. I'll be happy to. And, you know, just kind of like you asked, I actually am an engineer in my background. I studied engineering and I almost, sometimes I think I stumbled upon engineering, but when I think about it, both of my parents are engineers and kind of my family are all engineers. So maybe it wasn't such a stumble. I guess it's just like a family thing. But, you know, I started computer science and I worked as an engineer for many years, and just, you know, there was a time— you mentioned that I'm from Israel. I started as, you know, working in engineering in the military, actually. And then when I came here, I was at this company, General Magic, which is an amazing company. There's actually a documentary. You should all watch it. It's super interesting. I think it was coined as the most important dead company, but it was a privilege to work at General Magic. And at some point in my path at General Magic, the, you know, the engineering leader left. And, you know, I think like they probably looked around and the one, like, people kind of peeled off and I was next in line. And somebody suggested that I lead the team, which seemed a wild idea but exciting. So I took it on. So I kind of stumbled upon leadership in that there was like an opportunity and I took it. And yes, I started from engineering background and probably as a lot of leaders, I had this struggle in me for years of like, do I— am I more like engineer or I'm more manager? But, you know, I kind of at some point I decided that I'm more of a manager and decided to fully embrace that path of leadership and been doing it since. Just like you mentioned, I've been in all sorts of companies— enterprise. Now Intuit is more of a consumer company. Infrastructure— I was at VMware for years. That was— I was at Docker. So startups, big companies, infrastructure, consumer, enterprise. So I kind of feel like I, I sampled pretty much a lot of what we have to offer here at Silicon Valley.

That's awesome. And, uh, we're going to talk in a little bit about sort of the you know, how consumer-like things and sort of enterprise things sometimes converge and the experiences converge. And, you know, now enterprise experiences sort of feel very consumery. But I want to spend still some time about the path to engineering leadership and, you know, how you sort of went about, you know, as you transitioned into becoming a manager, you know, how you started to learn the different tools or tricks in the job that help you become a leader. And a better leader for people. And why don't you talk a little bit about, you know, were there mentors or how did you figure out like what were the important ingredients to being a great manager?

Yeah, you know, I think I made a lot of mistakes on the way. So I think it was like trial by error.

Me too.

Sorry, teams. For everybody that were, especially early in my career, it's an opportunity to apologize. But, you know, a lot of it is like, you're right. I looked at, I had different mentors and, people I kind of looked up to. And, you know, when I was thinking about learning from others, I divided probably my learning to two buckets. One is there's people that I was absolutely admiring, let's say, certain aspects of what they did. And there's also people I watched them and I was thinking, this is not very effective, so I should really make sure not to do this part. So, you know, but instead of just having one, like, mentor, I just really observed people around me and like picked what I liked and tried to drop what I don't like. But, and just like everybody else, when I got feedback from others more directly, often I was surprised in that some areas that I thought were like I was great at, they will show up as areas of improvement, and things that I thought I'm not so great, they will show as a strength. So sometimes you also need to— I guess my point is get feedback from others because your perception of how effective you are and what you do sometimes is slanted. So that's another thing that I've done, but lots of on-the-job working and like as a leader and like, you know, in any role you kind of have to be courageous and bold and try things and see what doesn't work.

Yeah, I always found the 360 reviews to be really enlightening. You know, I always— yeah, like you said, everyone has their own assessment of how they're performing, but there's lots of nuance or subtle things you do as a leader that sometimes you don't realize you're doing. Some of them are really good, and some of them, you know, people sort of maybe interpret the wrong way. And I always found those 360s to be just eye-opening for me and Even as, even as I grew as a leader organizationally, sometimes when you're managing a team of 10 and you have to change things when you manage a team of 50, right, or 500.

Totally. And you know what, the other thing I would say is every company is slightly different and every culture is slightly different. And you mentioned enterprise and consumer. I mean, those— the culture in those companies, they're different because the customers they're serving are different. So you also have to Just to say you have one bag of tricks and you never change is not gonna serve you well. You constantly, you know, you probably have a pillar of things that you lean on in terms of your tricks, but then you have to kind of keep adjusting and reading from your environment as well.

[Speaker:JASON M. LEMKIN] Right. So those sort of EQ, being introspective, you know, assessing yourself as you lead and manage others, those things are important, but what about being technical? Do you still think of yourself as a technologist? Are you technical now? Like, how do you stay close to technology? Then what about architecture and roadmap? Do you get engaged in those things? Is it impossible? Like, how does that work as you go from leading those small teams to now you're leading a huge organization and multiple product lines?

You know, my craft is engineering and my role is a Chief Technical Officer. I mean, obviously it has the word tech technology in it, sorry. So, I obviously, it's really important to me that I'm good at my craft. I just think it's very hard for you to lead technology and not be good at that craft. So, I actually think it's critical for any leader in any kind of rank, if you want, in technology to be technical. But obviously, I will do the service for my role if all day I just kind of sit on my laptop and hack, even though I would love to do that. So it's about finding ways to be technical, but I find it critical to be technical, so I'm looking for these things. So let me kind of share maybe a few things. First of all, I read every day. I read publications, I read news, I'll browse what's new on whatever kind of LinkedIn or whatever comes in, and I will go and read some of these, if it's a blog, if it's an article to understand what's coming. And then sometimes I go in depth, like if there's a new term or a new technology, I'll go and understand. So that's one is like just kind of keeping up to date of what's going on and what are the new technologies. Another thing that I do is, and my team knows that, is once a week we actually review in-depth different areas. And we rotate the topics, but different areas of technology in the company. And these could be pretty big meetings because we keep them open and people attend. But we go into quite a set of details there. Of course, we have demos and stuff. But sometimes, we actually open the code and look at the code. And while I wouldn't like— it's not like I'm trying to debug the code or that I can interpret it in the meeting. But the fact that we're going to go into details and people say, oh, this goes from here to here. Like, I can go really in depth in architecture, and that is important. And the third thing that I do, and I just kind of— my favorite is we have this concept of, you know, like similar to what other companies do, hackathons. We call them engineering days because for us it's not just about hacking. We're saying if you want to improve quality, if you want to do whatever. So these are days dedicated for engineering. But during those engineering days, which we have twice a year for a week, I actually block my time and I add myself to a team that has a project. Probably for them, I'm just like this big interruption that joins the team. But for me, it's a sheer delight because I actually get to code. Somebody will handhold me because often there's language that maybe I didn't program in or a set of tools that are new. But I'm just getting that experience of being in the code and working with the team. Aside from kind of refreshing my technical skills doing that, what it does also is it helps me observe how engineers work and what stands in their way, which I'm always interested in because I feel my role is to unleash the others to do the work.

It's so funny you mentioned on the engineering days, these hackathons, like how you would block off the time. I would do the same thing, and I also thought of myself as the interrupter or just like the distraction. But I think I, you know, those days were exactly what you said. It was a great way to learn about new technologies. And I always had, you know, like at Cisco, I think my org was, you know, a few thousand engineers. And there was always a couple that would always send me these articles or links to things. And I think they thought they were being annoying, but I actually found it to be a great way to like learn whatever they thought was interesting or sort of cutting edge. And yeah, I think those hackathon days and getting close to sort of, you know, people that have their ear to the ground, so to speak, has really been a great way. Yeah, like it's hard to do code reviews and look at code check-ins, but, you know, it's not that hard to follow along at least, right?

You know, David, I told you we're doing these reviews, and one of the things I've learned is to ask a lot of questions. First of all, I think it's good, just like a leadership technique is to, you know, you want to lead with questions, not with statements. But also, like, I've I got comfortable asking questions because, you know, we have a big span of technologies that we're using, anywhere from AI to cloud technologies to whatnot. And, you know, just kind of think of it, I could be like, have in-depth knowledge in all of them is probably not realistic. So I've learned to just be comfortable asking questions. And sometimes, you know, maybe there's a basic term I just don't know, or like an acronym that I'm not familiar with. Just be comfortable in asking. And just kind of— so I don't go with accumulating knowledge that has been— and it took me time to get comfortable because sometimes I thought maybe I should be the know-all. And I'm like, no, I can't.

That's exactly right. I want to shift gears a little bit and talk about how the tech landscape from a leader— we'll talk about technology in a minute and open source and those things. I want to talk about talent and, you know, how, how like sort of leadership has changed or how do you create an environment to attract the best engineering talent? You know, you've now seen this across a number of organizations and, you know, what do you think are the things that work best for, you know, creating an environment that attracts and helps you both attract and then actually hire and retain the best technical talent? I know that's a big question, so maybe we'll spend some time there.

Yeah, you know, you're right. The competition for talent is fierce. But, you know, a few thoughts in that area. You know, first of all is where you look for talent is an option for you as an organization. And of course, the bigger you are, the more you can. And today, I don't know, today everybody are virtual. So, you know, but like one of the things that we're doing is is we're not concentrating only in one area, but we actually are global. We have quite a few locations around the world. And, you know, what also this time where we all went virtual taught us that we can be even more aggressive. And you see those statements from companies, more aggressive of where you hire and tap to talent kind of in a bigger, you know, just bigger set around the world. So that's kind of one. So don't limit yourself to just places where competition is high. Is impossible, but just go global and go wide. So that's one. A second thing is, I think it's important to have like the most, probably the most important thing is the culture. And one of the things that when I joined Intuit, and I remember that I had kind of this roundtable introductory to the team, and it was just like, at the time it was just my staff. It was not a lot of people. So we go around the table and I asked them for their names and their roles and how long they've been at the company. So like the first person would say, "Oh, 4 years." The second one would say, you know, "5 years." And then people would say, "18 years, 20 years." And I was like, "What's going on at this company?" And one of the things that I've understood and learned to appreciate is that, you know, like, You need to really have a good culture and a consistent culture. And one of the things that I like about our culture is obviously we have this unbelievable mission of powering prosperity. And I think a lot of engineers, it resonates with them and you wanna work on something that you feel is good. And when you talk to, and I know when I was kind of early in my career and I was interviewing, I almost felt like silly to tell people that I want to work, I want to do like good in the world and just work on something meaningful, but it is really important for talent and for engineers. So if your mission is good and it resonates and then your value and culture are strong, that is really, that's like a glue and that's super important because at the end of the day, it's not just about, you know, making the money. It's about working on something that is fulfilling and the impact. And in fact, when people do surveys, they find that, you know, the compensation is not the most important thing. So that, that's of course is like, is important. So where you hire, just go global, make sure your mission and culture and values are all in check. And, you know, the third thing is obviously people want to be challenged, and just the way hiring is hard, Retention is hard as well, so just continue with programs such as mobility, change around, make sure that people grow as they do their job, they don't feel like they're stuck in a particular place. And then, you know, I would say the fourth thing is continue to compete, but, you know, make sure your compensation is good, make sure, you know, like you have a working environment that is is good and conducive to engineers that they can— we always talk about engineers doing the best work of their lives, but set an environment where they feel they can be productive, they enjoy their coworkers, they can contribute, they can move with velocity. So all these things. And I have a lot of other ideas about hiring, but those are probably some of the big ones.

Those are all really good. What about engineering recognition? And I was glad to hear that you didn't mention, like, you know, having free pizza on Wednesdays as being a definition of culture, which obviously it is not. But how do you recognize engineering talent and engineering teams? And, you know, what has been your experience with finding whether or not those things are important or not important?

Yeah, you know, and like, for me personally, I interviewed so many people and so many engineers throughout the years that you also develop a bit of, you know, kind of an understanding and a bit of, you know, like pattern recognition, even looking at resumes, frankly. And often, my team will joke that I was right on some of the talent, and sometimes it's talent they decided that, it's not good. And I'm like, no, no, no, no, no, like, you gotta hire this person, I see something. But, you know, in terms of— I would just say there's probably a couple of things. There's like the— you always want to look for like the hard skills, but I wouldn't over-rotate on that because, you know, like, as engineers, we always have to be learners and you always have to kind of learn. But, you know, if you have a history of track record of delivering, that's kind of super important. Like, what did you deliver? What did you learn? And I think not less important is your soft skills, right? And like, who are you as a person? Are you dedicated? Are you a hard worker? You know, there was this article, I think it was a blog, like years ago that, you know, I really liked, and it basically said that they, you know, you should look for smart people that know how to learn and that they can get things done. And I honestly look for these two attributes, and I add to that the soft skills. And I think that's really important because if somebody is smart, they will figure out anything. And then we all were working with, you know, technologists that they smart, they can theorize, but they can't get anything done. And that is a problem, especially as you try to ship products. And then obviously, like, who they are and are they a credit for the rest of the team, because, like, you know, engineering is a team sport. So you have to kind of work well with others. Are they— do they care about the customer, you know, or do they just want to work on, you know, technology? So those are some of the attributes that you will look for, but smart, getting things done, and have the soft skills that you want.

Yeah, those— it took me a while to put those soft skills on equal footing with technical skills. I think that I learned the hard way that, you know, when you have a great engineer who does just, you know, incredible work but causes a ton of collateral damage with the team, you know, maybe is insulting other people, committing on top of their code without any conversations, like the damage from that ended up being, at least in my experience, painful to manage that eventually we had to let that person move on to another organization. And it was painful because they were a great engineer, but just too much pain and friction from other great team members.

Totally. And sometimes you're like afraid of like what happens because they know everything, but you realize that, hey, actually the team is much better now.

Totally. Yeah, those keyboard commandos, like, yeah, they're powerful, but You know, maybe when you're only 5 or 6 people and it's just a small, tiny startup, but in a functional organization with real deliverables, it just falls apart and causes pain. We are getting good questions coming in, so I'm going to ask some of the questions coming into the Q&A box and just want to remind people to feel free to put questions in. I'm going to start asking Marianna some of them. So one of them is about cloud, and I'm going to ask preface question before I ask the question that, uh, that Chris asked us, which is, um, in your job at Intuit, are you using public cloud or using cloud technologies? Um, tell us a little about your, your tech stack to the extent you can.

Yes, you know, we actually, uh, went through an enormous, uh, migration effort, and you've seen us probably speaking at AWS, but we are an AWS customer. And we've been using AWS. So yes, we're very much using cloud technologies.

Okay, so have you ever been— so this is not getting to the question, you know, as you look to cut costs, have you ever— and I know, yeah, you know, you're a big AWS partner, and like you said, you've been on stage with AWS before, so the answer is probably no. But have you looked at sort of multi-cloud or using other clouds for other services, or even running your own cloud? Have you ever done those kind of analyses, and how did it shake out?

Yeah, you know, on kind of running our own cloud, we came from our own— like, we had our own data centers and everything. So I think today, if, you know, for most companies, and just kind of, again, just speaking from being in the infrastructure space for quite a bit, unless you have very specific needs, the public clouds are offering some unbelievable infrastructure at an unbelievable cost. Clearly, you should think many times before you build your own cloud because we know how complicated it is. David, you come from an area of infrastructure and equipment and this, and it's a lot. It does make sense for some companies, but for many companies. And this is why those clouds are becoming so popular. But obviously, we always scan the market of not just cost, but features and whatnot, and understand. And we made our choice very selectively and very diligently. So we looked at quite a bit. And in terms of cost cutting, I would say even if you are on a particular cloud, it's not It's not always that you need to go to another cloud to save costs, but there's probably ways in which you can optimize your usage. And we've been going through a journey, and this is where, again, I have to say that AWS has been a great partner in trying to understand what is the cost and how we are optimizing that. And there's a lot of— ways. Like, you know, do you really need to save the data? The data, as long as you save it, do you— are your CPUs need to be running when really nothing is going on? We have processes running where there's nothing. They're waiting for, you know, for, for stuff to happen. I'm like, no, like, you don't need them up. You know, there's, there's different ways to save. So I would say whatever cloud you end up with, there's like a lot of ways in which you can save after you are in that environment and look for them. And you talked about open source, and another way to save is think about what technologies are you bringing in, and are you leaning on open source? So there's lots and lots and lots of ways to continue to save in whatever cloud you're in. And every year, I'm thinking, oh, we might be maximizing our savings, and every year, we find more prizes.

Yeah, I think that's— I think every— maybe not every, but a lot of engineering leaders have, you know, every once in a while, they'll get this email from somebody on the team saying, "Oh, you know, we figured out this way to either, you know, buy reserved instances of guaranteed capacity, and now we're going to save $2 million a year," or we found some performance improvement using containers or something that allows us to cut our number of instances down by a third. And, you know, I remember I'd get these emails, and on the one hand, I'd be like, wow, this is awesome, this is incredible. And actually, one of those emails once came after a hackathon, so it wasn't even part of the engineering roadmap. It was somebody who had sort of thought that there was a way to, you know, better allocate resources and ended up saving millions of dollars. And, you know, I think your point is well made that there's a lot of optimization to gain from your current cloud provider if you really dig in and not just look at the superficial pricing. And, and, uh, and I think a lot of, you know, in case people who are listening don't know, if you are using a lot of compute, you should have a direct relationship with the, with the cloud provider, and there's a lot of better pricing than they advertise on their, on their site, just like most enterprise software.

Totally. And David, I'm sure like in your new role as a VC, you'll probably see a lot of companies, small companies that have all sorts of ideas and of cost optimization, and it's worthwhile to also look at what's out there and how companies help you save costs.

Yeah, that's right. So a bunch of questions have now been coming up, and I'm surprised actually. A lot of them are much more about managing people than they are about making technology decisions. So I'm just going to start digging into some of them. And so the first was, how do you focus on motivating your technical staff, especially staff that has been at the company for maybe 10 or more years and keeping them engaged and motivated to perform their best?

So, you know, it's funny that you mentioned there's a lot of questions about people. I always think that we technologists, you know, we love to talk about how we technologists, but we always kind of struggle. It's not struggle, but you know, like when you're in leadership, like you just, you're gonna have to focus on people and just so important. Sometimes I think we underestimate that.

Nobody comes to us with compiler problems. It's always people—

No. Nobody asked me a compiler question for quite a bit. So, you know, on how to motivate people, again, it starts with the mission and making sure that you start from like— that you have a clear mission for not just the company, but the team. There's clarity of role. I just find that sometimes when there's no clarity of role or what you're expecting, and not just kind of what you're expecting in terms of performance, but what is the people expecting, then it just clouds. It's like people are not sure what's like what they're supposed to do. So they kind of can retract to working on like a smaller problem. So I would say clarity of role and clarity of like, here's the mission. And you need to be clear for yourself. Just like because we have a mission for the company, I had to spend time and worked with the rest of our leadership team and our CEO to kind of think about what is the technology direction and really spell it out. So, it's super clear for everybody. Here's where you're going, and here's what we're trying to do. And then here's your role in that. And here's how you can make impact. In my eyes, these are like the biggest, things that motivate me and that will motivate the team is that knowing what heels you're after and kind of what is your role in capturing those heels, that would be like number one. But then second one is make sure you have an environment which is really, you know, people, they can be themselves. They, you know, a lot of conversations now is about not just kind of inclusion, but actually a feeling of belonging. That people feel like they belong here, they can be themselves, they can speak their mind, you know, so that you can really bring the best out of them. And that's true for people that have been there for a while, the people that are new, and, you know, so just make sure you have that environment and lead with energy. I try to lead with like energy and humor, and, you know, partially I like to keep things light and engage with humor and Honestly, I have to do it for myself because otherwise it's like, I'll be like, oh, I just can't have all this like slides, slides, slides, meeting, meeting, meeting. So, you know, so for me it's like, you know, okay, well spice it up with some fun and just make it light and know how to laugh at yourself and how to kind of tackle things with seriousness, but also with lightness. You know, so those are some of the things that, you know, I will also say like, just make sure that also you have the right talent around you. And that people still have the spark in their eyes as they think about the customer problems, etc. Because who is the team around you is important. So those choices and motivating the team is important. But at the end of the day, one of our biggest roles as leaders is to make sure we have the right talent around us. Just, I would say, make sure that the talent around you is, is right, and then you kind of do all these different things.

Um, excellent. Um, and so one of the interesting things that sort of keys off of that answer is there's actually 3 or 4 different questions that are all asking, um, essentially the same version of how do you measure results, or how do you measure productivity of teams. And one of the questions is actually specific about how do you measure productivity in a remote world? And I have some thoughts on this, but I'd love to hear your, your views first.

Okay, I'll start, David, and you'll continue. I actually think that remote world is helping us measure productivity. So first of all, I think as many companies have noticed, I mean, we thought that our productivity numbers are gonna tank, and sure enough, at the beginning they did. But I think a lot of conversation, like I've talked to several colleagues and everybody sees that not only productivity numbers are steady, some of them are going up and it kind of leads you to asking, hmm, what kind of overhead do I have? And of course, there's also the sad reality that some people, they just kind of bored right now at home, so they work more, but that's not how we wanna get the productivity numbers up. That's not a good side effect. But the measuring, first of all, I'm a big, and this is one thing that I think is so true for consumer world in general, and is different, that's something I had to learn that is different than enterprise. And David, you might resonate with that, is data has different relationship in enterprise and in consumer. In enterprise, you have a customer, like we always knew who were the top 10 customers. They had names. I knew those leaders and those other companies by name. We have each other's cell phones. They will escalate to me if there was a problem. And so, there's a lot more of a relationship with your customer. But in a company where you have millions of customers, you can't develop relationship with millions of customers. So, you rely on data a lot more. So, you know, like part of any DNA of any company, but in particular consumer, is like you have to look at data. So data speaks the truth. I mean, if you look at the right data, 'cause you know, there's also topics of bias of data, et cetera. But, you know, like, so look at data and, you know, as a measure for productivity is super important because if you don't know where you are, you don't know what to fix. And if you don't have— you can't fix what you can't measure, right? And I would say for productivity, boy, this is such a hard set of topics because how do you measure and what is exactly to measure? And we had different metrics around productivity and we actually now trying new sets of metrics, but let me tell you, how I think about it. First of all, I prefer to measure productivity as in what is outcome that gets into the customer's hands. So one thing that we are measuring is deployments to production and the frequency of those deployments. So that really helps us understand of what actually gets in customer's hands, because at the end of the day, everything else is just like an, you know, other sets of like metrics. Then inside that, I would actually say it's important for me, at least, that we measure what breakages we have in our system, as opposed to try to measure our engineers productive. I think we have other ways to evaluate performance. So I'd rather we look at metrics that help us understand What are some of those blockers, breakages that we have in the system, rather than say, are engineers working well enough? I mean, we have ways to measure performance. So I'd rather we understand whether we have the conditions to allow great velocity and that we fix those problems. So that's kind of another thing. So, like, when you take that final number and then you think about all the steps along the way that software making and trying to understand our system fast enough. Is there enough automation? Are we merging at the same pace that we're creating? So those are some of the things that, you know, and obviously GitHub has a richness of metrics you can get there. So those are some of the things that we're measuring. But I would say start from the metrics that are right for you and then look at the chain, look at the most important metric, and then have the chain of metrics to get there. That helps you with diagnostics of like, you know, what to fix. You know, and in terms of like working virtual, obviously like we can, you can look at the conferencing software that you use and you have a lot more insight of the amount of time people are spending in meetings, which I know is often a complaint in a big company. And like you can start looking at that and see if you can optimize that.

Yeah, I think my observation, at least in looking out now, I work in venture capital, so it's different, but I work at a big venture capital firm where we have 200 people, so it's a little bit like a company, is that we actually, I think we have a lot less meetings because we're not in person anymore. And so I feel much more productive because there's just less meetings. I don't know if that has been your experience or not.

Yeah, you know, less meetings, but obviously we still have a lot of, you know, video conferencing meetings. And again, there's more conversation on this fatigue of these meetings. And honestly, in this video conferencing, you kind of stare at yourself all day because you see yourself as sometimes a big image, a small image. And it's tiring. So I would say that, yes, there's less meetings. There's also less of these corridor interactions, which some of them I miss because I would get so much done just going from one conference room to another, I'll meet like 10 people along the way and we'll be able to resolve a lot of topics. But, you know, like on the other hand, you know, you don't have this kind of constant action item, so you can focus on like certain things. So yes, there's like pros and cons, but we definitely— our usage of video conferencing, our usage of— we use Slack, so our usage of Slack went up. So, you know, all of these are ways in which we're trying to compensate. But yes, there's a lot less interruptions for sure, and a lot less time spent in meetings.

So there's still a lot. That's fair. One thing I've heard companies doing to try to recreate the water cooler kind of effect a little bit is, you know, for their first meeting on Monday mornings, which is usually sort of like a staff meeting or a team meeting, you know, people will join a few minutes early to sort of get to kind of like walking into the conference room a few minutes early just to catch up on, you know, how was your weekend, or sometimes talk about those kind of hallway conversations. And yeah, on the measuring productivity, I'm hopeful that in this remote environment, you know, a little bit like you said, that managers and leaders go— instead of measuring inputs of like, you know, how many code check-ins maybe did you do, or when did you show up to the office and when did you leave— instead of measuring those kind of inputs, which are fairly useless I think they go to measuring those outcomes and those results. So when do things get to customers, number of major breaking bugs or things like that that are really much more results and outcome oriented. So I'm an optimist that some of those things that I think become best practices in a remote environment will just turn out to become best practices in general, not just a remote best practice. So we have about—

Totally, you said it much better. Measure the outcomes, not the inputs.

That's right. So we have a few more minutes left here, and I want to make sure— one thing is, look, you are, you're a woman in engineering, a leader. I'm sure you've seen all kinds of good behavior, bad behavior. You're an inspiration to lots of other leaders, men and women alike. Why don't you talk a little bit about, you know, how you think about that topic. Is it an important topic? Is it top of mind? Or is it not a focus area?

Well, listen, obviously diversity, inclusion are super important topics and ones that I'm personally invested in. You know, we as a company invested in. I don't think I would have been able to work at a company that don't think it's important. So super important topics. You know, the way I always think about those things, David, is I try to ignore, kind of not make it a topic. So like, I always rather just show up as a technologist and talk on technology as opposed to kind of spend time on talking about the fact that, you know, I'm, you know, a woman in technology. But I think it's a super important topic and one that companies should take very seriously. And again, we talked about data. You've got to look at data and you've got to understand your gender diversity, your ethnic diversity. All of these are super important. And if you don't know the numbers, you're not going to know where to go, like what to do. And a lot of people emphasize hiring. Hiring is important, but the environment you create is so important. So you have to look at also other metrics. Are you able to retain your talent? Is your talent— are you able to promote them, you know, is there like— it's super important. So look at data. It is absolutely a topic, uh, that is important. And, uh, um, you know, just, just kind of, uh, continue— let's continue working on it as an industry. We definitely have, uh, still ways to go.

Yeah, there's lots of, um, I think a lot of work people do rather than just talking about our programs. They create mentorship, uh, opportunities and, uh you know, leaders on this, on this call and future managers, you know, there's a lot of small things people can do to create more welcoming and inclusive environments and then support people that, that, you know, are from a more marginalized community that are trying to enter the technical workforce. And so I'm glad you, you focus on those things and speak about those things. Is there anything we didn't, that we didn't cover in our last couple minutes here that you want to make sure we cover? This has been a really interesting discussion. And we really appreciate your time. I want to make sure we cover everything that you wanted to cover.

I know, I think we touched upon everything, but I would just say, we talked about the importance of staying technical and kind of monitoring trends. We didn't talk about trends, but obviously, like, I've been monitoring what's going on with AI, what's going on with devices. So lots of those kind of trends, We talked about leadership and the importance of, you know, your principles around your leadership and how you motivate your staff, how you hire. And then, of course, you know, the importance of data. And, you know, we've been relying a lot on data to understand where we stand and also what do we need to fix.

Awesome. Well, I think that's a good— I agree with everything there. And I knew I was going to agree because you and I have spoken before. And you're a well-respected and well-recognized leader and technical leader and just business executive in the industry. So I think we all thank you for your time. We appreciate you taking your time out of your day to speak with us and to be here at SASter with us. So thank you so much.

Thank you so much, David. It's such a pleasure. I really enjoyed our conversation, and thank you for anybody who listened.

Likewise. Thanks.