Building a Stellar Engineering Organization

Please welcome scott bechuk, nvp partner vijay gill, databricks, svp of engineering, weeping peng, salesforce software architect, and claire huff, Apollo vp of engineering. All right, well, thanks everyone. I appreciate you joining us for this session. This is going to be arguably one of the most energetic sessions of Saster, and so I'm sure you won't be disappointed. We are very lucky and fortunate to be on stage with some very talented engineering leaders. And you can see with their backgrounds, we've got a lot to talk about here, so let's jump right in. So, Vijay, Claire, you've built some incredible engineering organizations in your careers. What does it take to hire and retain the best engineering talent in the world?

Oh, boy. We could have started off with an easier question. So, a couple of things I want to kind of point out here. One is, especially at a startup trying to compete against Bay Area offers from Google, Facebook, Amazon, LinkedIn, Twitter is playing to their strengths. So I always thought that we should play to our strengths, which is try to play asymmetric hiring strategies normally focus on things like differential impact. I tell this to new college grads all the time. There's like 10,000 of you joining Google every other year. Whether you join or not, nothing's going to change. At Google, for example, great company. I used to work there. But you join here, you'll be like a measurable fraction of the engineering Org. So your ownership and your impact and scope, you will grow faster. So you'll have more differential impact. The second thing that we've done is go compete in a geographical, distributed fashion. For example, there are certain countries, for example, where a lot of these large companies aren't there. So try and build up engineering offices there, Claire.

So the strategy that I've been taking lately at Apollo is actually we have a distributed teams. We have half of our engineering team here in San Francisco, but everybody else is all over the world. I have an engineer in Mexico, Amsterdam, Finland, Australia, etc. So you find talent where you can. And given this day and age, with all the connections we have and all the way to communicate with your teams, there's no reason why you have to hire here. And when you get the right candidate, you have to speak their language, whatever their language may be, because engineers are motivated by different things and you have to find what is it that's motivating them. And when you can match that motivation with what you're working on, that's when you can actually be competitive and make the hire. So as a small company, it's always hard for us to hire. If the candidate's going to go to Google, Twitter, Airbnb, they're not going to come to us if that's their goal. So we need to look at what's their motivation, what's your goal, what kind of impact do you want to have and what motivates you? I was at Udemy up until about six months ago. I was there for five years growing the organization from 13 engineers to 150 engineers. And we built 25% of the team in Dublin, Ireland and the other 25% in Ankara, Turkey were their top three technical universities. So you have to kind of figure out what is your strategy. There's and Udemy is an e learning marketplace. You should all be taking a course on udemy. They have SaaS offering your company should be signing up. So Udemy our mission was we improve lives through learning. So that mission, if that speaks to somebody like I want to be part of that mission, then the hiring gets a lot easier. And then you need to kind of speak to what their motivations are, how we can leverage their skills skill set to really build our platform.

Yep.

So you said something in there that was really interesting. I want to go off script a little bit here. You mentioned remote teams and I think you know whenever building engineering a lot of different orgs in tech companies, particularly in modern tech companies. How I ask Weiping because you're with Salesforce and Salesforce deals with remote engineering teams both developers and engineering management in remote offices sometimes in time zones that are on the other side of the clock. So what's the best way to communicate and keep the right cadence with remote teams?

Well that's a tough question but at the same time I think it's actually there are many ways to deal with it. One thing it's almost like a culture wise make sure you are very inclusive. When people are remote they're disadvantaged for sure because when you have hot composition all that you might be just missing that one piece communication. So don't even worry about there's so many different channels nowadays Slack and Salesforce has a chatter and there are many different tools for different way of communicate and I think communication is the key make sure you always have the other side of your team informed and sometimes go have a little fun get people together and face to face time helps a lot too.

What is the one biggest fail that any one of you has experienced in dealing with a remote team? What is the one thing you always want to avoid?

Oh boy, so many failures. Kind of hard to catalog. There's two or three main failure modes I've seen and experienced over time. Large companies, small companies, doesn't matter. Number one thing is, I think terminology matters. I learned this lesson. Never call them remote, because remote implies they are like remote from some sort of center of gravity. So I always term them distributed teams. You're a distributed team, which kind of balances out the verbiage weight of being remote. They're already far away. They feel disconnected. Calling them remote just kind of in the taxonomy itself puts them at a disadvantage. So from a taxonomical perspective or an ontology of teamwork, I always call them distributed teams. And it takes a lot of reinforcement with my engineering leaders to stop using the R word. So that's number one. Number two is people tend to use offshore or distributed teams as infill. Right? Okay, I don't want to do this maintenance work. Hand it over to the distributed team and they can work on it. Communications, to Weiping's point, is massive. If you just give them crap work, you will eventually end up with crap people. That's just how it goes. Because anybody who's good and ambitious, they're not stupid. I assume you're not hiring for stupidity. If they're not stupid, they're going to find out they're getting the work that is not going to be materially helping their careers. They're either gonna transfer to your office or they're gonna leave. Ergo, over a period of time. What you'll end up with is essentially people who can't leave anywhere, or people who are demotivated and just show up for the paycheck. So the productivity starts falling. And that's one of the key things you want to. Want to track. So the things that I've done, which is on the positive side is like make sure the taxonomy is correct, make sure you over communicate, make sure you give them accountability end to end. And this is going to be painful because you're so used to everybody being local that when somebody is distributed and you give them autonomy, you're like, oh my God, they're gonna like mess it up. So give them like small, insignificant work. And this is where you, as a leader, you have to kind of step over that chasm of giving them stuff that is actually material and then. And they'll mess it up. Absolutely for a couple of times. It takes about a year to 18 months to get a working bootstrap distributed team on a small component. Start small, make sure the interfaces are narrow, make sure that the interfaces and APIs are clean. The Money you save, you spend on.

Flights back and forth and then you have fun too.

Yeah, good point.

So let's bring it back home then. Let's talk about it doesn't matter where your engineering leaders are. And let's talk about though and I love to oversimplify things because it makes it harder. What is the one skill that every engineering leader, whether IC manager, executive, what is that one skill that every engineering leader must have? Let's start with Weiping.

Okay, well if you have to say just one thing, I would definitely put it on stay on top and stay, stay alert on all the different technology trends. Technology has been changed for so much and the development is very amazing. Like if you think about a few, probably less than a decade ago, everybody running your own data center and all that and then just a few years Amazon and Google, they just all the public clouds, everything is there. If you stay to say that I like my way without knowing other people has been very quickly developed new different ways you could be falling behind. And there's so many open source library and a lot of communities, they're building many different fancy stuff. You cannot single handed build everything yourself and try to stay on top of it and be aware what you can leverage, who you can part and what kind of open source library you can borrow. Whether you are an individual contributor or a technology leader manager, be able to speak to us and understand what impacts your business units and in the end of the day it's about cost to serve.

Yeah, well what I'm hearing there also is nobody knows everything. Keep your ego in check and always be willing to learn and keep an open mind. Claire, any other ideas?

Yeah, it was actually very similar. We chatted about what's most important to me it's growth mindset. So when you have growth mindset, you're always learning, you're open to learning, you're open to admitting that you don't know something like you said. And I think that's super important when technology is changing all the time. And I've also gone from startup to startup doing very different things. So you have to be really open to taking on those different challenges and also learning from different types of people. We have a very diverse community of developers and we need more diversity. So you need to be constantly learning about not just the technology but what motivates people, what are the backgrounds of people that you're working with. And in order for you to be a motivator, in order for you to set an example of being a leader, you have to be open Minded. Be able to say you don't know something or when you make a mistake, be able to admit your mistake and reset and make progress.

Yep.

Vijay, any other thoughts?

Well, they said everything I was gonna say, so I'll add one more component which is actually, I think, underrated empathy. You must be able to empathize with your folks and, and why they're there, what they're doing. This goes back to Claire's point. Connect them to the mission, connect them to the vision, empathize with them. They have challenges, you have challenges. Make sure that we don't go into a us versus them management as a.

Leader and yeah, easier said than done, but I understand, I agree with you.

We're all imperfect. Right. We have to try to not mess it up on a regular basis.

So the next topic I have is a controversial one and I love this topic because we just happen to be sitting here together with a team of very senior engineering leaders. I want to see in the audience, do we have any product people in the audience? Okay, good. Do we have any UX people? Designers? Okay, a few. A few. Okay, so let's talk about this. Let's talk about this. What are some of the best practices for related to how an engineering team should work with and communicate with product and ux? And there's more buried in there and I'll get to it in a second, but let's start with that.

Okay. Well, I think the first thing coming in my mind is always remember you are one machine. Or you can say, I'm just one body. Your left arm versus right. A lot of times engineers can geeky out. I can make this go so much faster if I try this method. And product manager would be like, if you build that one thing can be stupid, but customer loves it. How do you reconcile in the middle UX the same thing. If you make this one click, right click, left click, it all make a difference. It can.

Wow.

A lot of usability stuff too. So in the end of the day, I think what we really need to be understanding is we're developing one product and makes money or not, getting the market or not, whether people are going to like it or not is the most important part. And if you keep that in mind, you will be willing to hearing each other out. And somewhere in the middle we will get to a point to say maybe next release or maybe this release.

But in a situation, I'm talking to you now, Claire. In a situation where engineering, product and UX all have really solid points, points that make complete sense from A customer perspective, maybe from a technical perspective, maybe from a market perspective, everyone's got a great idea. Is engineering always right?

I actually I've been a product manager because I was hired by a founder who wanted engineering minded or engineering driven product organization. So I ran product for a little while alone and then I ran a combined organization. So in my last few jobs I've been calling. We never distinguish product design and engineering as separate teams as much. We sit together, we call ourselves pde, Product Design and Engineering. We do one status report and you know, sure there will be some disagreements about like how we might do implementation or like what technology we might adopt and all that but. But at the end of the day our end goal is to serving our customers. I think if you anchor on that and really doing the right UX research, you know, really listening to your customers, it's difficult to come at this like a lot of differing opinions. And I would say let the people bring their expertise to the table as a team. If engineers know how best to implement that, let them do it. If product manager knows what are the competitors doing and what do our customers want and they're bringing that to the table, let them own that and really drive that within the team. If designers know how to actually design the product, that will be easiest to use and it will really satisfy our customers. Let them bring that expertise. It's about bringing that great talent together to make that great product.

Good points. I'm gonna put you on the spot, Vijay. I know the last three companies you worked for have been incredibly successful and you've been in roles where you've had to directly interact with product in those situations. Tell us a story, tell us a story of a scenario where you where engineering had a fundamental disagreement with product and how did you resolve it?

Oh boy. Thanks for that, Scott. Well, I mean I don't think we need to go too far back. One of the things we constantly run into is, I'll give you one fundamental disagreement is stability, availability aka go to market and expansion versus new feature and landing new go to market. Space 1 is land and expand. So that's like things like enterprise features, security, availability, reporting, audit logs AKA the go to market. People love that. Right? Because they're already in the account. They don't have to do new customer logo lands. They know exactly who the champions are and how to build that in. But you also don't want to be like looking backwards. You also want to acquire new logos and building new logos and acquiring new logos sometimes require new capabilities or features of Product. So the massive disagreement we have here is depending on how strong the go to market team is, you spend a lot of time on what I would call RAs, reliability, availability, security and stability features and how much engineering capacity you carve out for those versus engineering capacity for keep the lights on, like interrupts, escalations, break fix, versus how much capacity you reserve for for new features, new products. And so that balancing the tension is actually painful. And so the way we've been doing it, to put it very in concrete terms, is that all PRDs, we do joint PRDs, one engineering leader, one product leader for the PRD, the product spends a lot of time inbound and outbound work. Then they take those lessons learned. We write a joint prd, then the engineering team does joint capacity planning. So we reserve some capacity planning and we have it documented because, well, we are a data processing company. So I know exactly, for example, that about 30 to 25% of every team is what I would call KTLO. And then we carve out the remainder for features and for RAs. And the RAs is our champion for RAS is go to market. Our champion for new feature expansion is product.

When you all get together to write that joint prd, how thick is the padding on the walls in that room?

So far it's actually working out because I think to Claire's point, if you anchor on the customer, what does the customer really need? It kind of instead of going like this, head to head, like I'm right and you're wrong, or you're right and I'm wrong of a zero sum game, you kind of like pull back to the side and then you look at the problem in without putting your ego in the way. And of course, if you have revenue numbers with the problem, makes the argument somewhat easier to tiebreak, I might even.

Add a little more. So I do believe that there are three arms, three pillars in this whole machine and you want them to really talk about the different things. Sometimes even getting into arguments, it's very healthy. And if you see people don't argue with it, there's actually a problem because there are different things. Exactly. Like Vijay was talking about the technical debt. Sometimes it's so big there, if you don't, as a technical person, if you don't speak up from the business perspective, they don't see it. They see we sold it, customers sign up for it, and when this big mention is going to collapse, if you don't say it, nobody will know. So it is your job, everybody's job, to speak up and Debate all fair points.

I want to continue, though, talking about. We've been talking a lot about things that mature engineering leaders do. Okay? So in a perfect world, we're all playing nice together. We all have had the experience that this collective group has had, and we figured out how to make it work. But let's talk about engineers who are just starting their careers because we're all hiring in our companies, engineers at all different levels. I want to know what y' all think it really takes to bring an engineer from just fresh out of college, getting the first experience. What does it really take to grow from an engineer to what you would consider to be an engineering leader? Maybe we'll start with Claire.

So I've been recently exposed to Apollo's an amazing company, and they hire a lot of college grads from top schools. We're talking mit, Carnegie Mellon. And I'm just so impressed with young people these days. They're just super motivated, right? So how do you grow them in, like, becoming a leader? And I think a lot of them just come and they want to soak it up. They want to learn as much as possible. And sometimes some of those skills just don't come. Just being super highly motivated. You got to be exposed to, you know, different problems over time. So I think there's a little patience that is required. But, like, some of the things are really difficult to teach. Like, Vijay talked about empathy. I think that's. Those are. Empathy is actually a skill that you can teach. I believe you could teach compassion. But, like, they need to be exposed to what it means to have that real empathy. They need to understand that people come from different backgrounds and people have different challenges, and you have to work with them and find the right motivation for people to become that leader. So I, you know, I. Part of, I think growing young people into leadership is exposing them with as many problems as you can, but also helping them deal with difficult situations that allows them to expand more, you know, their diversity of skills when it comes to, you know, it's not just the technical skills. You know, it's not being exposed to different technical problems, but also different people situations as well as customer situations.

Vijay.

So a couple of things, again, to make it very concrete. What we do is in our job ladder, we have a job ladder, which I wrote up a couple of months ago. We have explicit gates, like, for example, if you're like level three, which is NCG, or level four, which is new college grad, PhD. There are certain things and expectations that we do. But if you go up like for example, at senior, you can start authoring design docs. And to get to senior and authoring design docs, all seniors must be able to mentor a junior person to write a design doc, for example. And then as you get to senior staff or principal, you can then kind of do things like, okay, I want an autonomous group to run with this idea and you get to design and you get to architect the entire thing end to end. So making it explicit on a job ladder makes people kind of like, see how this progresses. That's number one. Number two, mentorship and pairing them up with good strong leaders. And the third thing is not everybody's going to be a great leader, right? So you want to figure out you play to their strengths, not try to make everybody the same. And that's how at least we've been managing to do it. But the biggest differentiator is who they're pairing up with and the opportunities they get to learn. This goes back to taking sometimes a risk. You have to push people slightly beyond their comfort zone to get them to grow. And sometimes when you push them beyond their comfort zone, they will be failures. That's why you have a more senior engineer to kind of catch them. Wonderful.

Yeah, mentorship. I mean, I'm hearing a lot about mentorship. I'm hearing a lot about teaching the more difficult soft skills, things like empathy and working within teams. All very important. Weiping anything last to add there?

Well, you two covered a lot. I think those are really good advices. One thing I can speak from the technical side of it, it's actually just like how it reminds me, how do I do technical interview? Sometimes it depends on the person's seniority. The question can start with just about any question. You can start with very simple and you can always have more things to add. So usually the way I, if I were to mentor a particular junior engineer, usually it's give them some ownership so they feel that's something they own. However, however, you can always modularize the whole problem, give them a very specific one or for more complex one, just gradually add to it. There's always more to add to the complexity. And it's amazing. People actually learn. I've seen a lot of engineers just grow very quickly. They have the hunger, they always wonder why. And sometimes you'll be amazed. Before you ask the question, they ask you this, what about that? So yeah, it's a pleasure to work with a lot of those amazing engineers.

Okay, this has been great. We have time for one last question and just to kind of wrap Things up. I would like to ask each one of you. We have. This is, by the way, Saster is a great community of entrepreneurs, of investors and of people who just love building great companies. What is the one thing that you'd like to leave everyone in this room with when it comes to building a great, great product and technology? Organization? What is the one ingredient? It doesn't have to be the only. Let's just choose one. Anybody can start.

So I've done a lot of startups, so maybe some of you are entrepreneurs and you're at a startup. I think startups are made of people with grit. It's hard, right? Startups are hard and you know, you gotta really like dog it every day with different problems and you know, like trying to figure out where the product market fit is in the early days and then how do you scale your operation and actually serve your customers? All those things will come in time, but you gotta stay in it. Like you cannot have 90% of your foot in, you gotta have 150% in and you gotta tackle each problem that comes. And again, like I keep saying, you gotta stay in it. So I really encourage everyone to go into a startup with that resilience and that any problem that you will face, that there is always a solution and you'll figure that out. You may not know everything in the beginning, but with that, with that determination that you'll figure it out, I think that is key ingredients for entrepreneurs and people who are joining startups.

Motivation goes a long way. What do you think?

Definitely.

I think number one factor is try to be lucky. If you're going to be lucky, luck strictly dominates over every other thing that you've heard today. If you can be lucky, at least try to set up success factors for success. And specifically, each team that I build has to have five ingredients. Mission and vision. Somebody who's good at execution, somebody who's good at tech lead, somebody who's good at product, and somebody who's good at engagement, HR and people. If you have those five elements on a per team basis, you will probably be less likely to fail. Although the default state of any startup is dead. And this will make it less likely, I guess.

Wow, that's very uplifting. Great way to end.

It is true.

Not untrue. Not untrue. I agree. Weiping.

Okay, well for me actually I have not started a startup company before. However, knowing I've been working with so many different big companies, I do see it doesn't matter how big, how smart and organization is one thing I always learned is work with, Find as a group of people, there's a few. It doesn't have to be everybody, but find the group of people that you are very tight, you always can trust, always can rely on, and they might find the other ones. So together, you build that tree and get all the things done.

The commonality here is trust. Trust the people you work with, trust the mission you're on, be willing to put, put in the hours, the hard work, the motivation, the grit, as you said, Claire, I want to thank our panelists here. Let's give them a big hand.