Growing Product & Engineering Orgs with Twitter VP Engineering + Redpoint Ventures Managing Director

Okay. And you can begin countdown now.

Welcome, everybody. My name is Tomas Tingus. I'm a managing Director at redpoint. And I'm thrilled to be joined today by Nick Caldwell, a friend who is currently VP of Engineering at Twitter. And we are going to be talking about growing product and engineering team organizations from zero to ipo. I'm thrilled that Nick is with us because he's got an incredible background. Nick grew up in Maryland. His first computer was a Tandy 1000. I had a lot of fun researching you, Nick, by the way. And you set up your own BBS on that machine. Then you went to a magnet school, you rose up to the ranks of mit, and you started working at Microsoft. And your first car was a Nissan 350Z. What color was it?

Oh, it was silver. Man, that was amazing. I bought that car before I even got my first paycheck. I'm not lying.

Is that true?

Wow. Yeah, I was making bad decisions back then.

Then you rose up through the ranks at Microsoft. You started the Power BI team, grew that to 300 people. Then you switched from B2B to B2C, which is rare to see. You became the VPN of Reddit, which is the front page of the Internet, and you tripled the engineering team there in 18 months. Then we got to know each other because you became the Chief Product Officer at Looker, which then afterwards became. Was acquired by Google, and Now you're a VPN to Twitter. So you've seen both B2B and B2C. You've seen incredible scale. You've got a wonderful perspective on all this. And I also learned that you love video games and sneakers on the side. And it's clear you're a LEGO fanatic from your background too.

Oh, yeah. I had a lot of free time between my current job and the last. So we put the Millennium Falcon together. But thanks for the intro, Tomas. I was looking forward to this Fireside chat for quite some time and really appreciate you inviting me to the stage here.

No, thrilled to be having the conversation. So let's jump into the question. So, one of the things that I really wanted to talk to you about was when founders are starting companies, the first goal is really the establishment of product market fit, making sure that customers actually want the product that you're building. And then afterwards, there's this transition from innovation to execution. So now that you've built the product, you actually want to start scaling the org. You've made that turn several times. The big change that you've written about in the Blogs is sales and marketing now starts to influence the product roadmap. What does that mean? What does it feel like in an org? How should you as a leader within a startup, sort of anticipate and expect? What changes should you expect in the org when you make that transition?

Yeah, you see a lot of companies struggle with this. And I think that blog was written in the context of Looker, which is a company. I in part came to help this specific problem. But early on in your product life cycle, you're obviously in an innovation phase. You're in that sort of random walk trying to find product market fit as quickly as possible before your funding runs out. You have to get a different job, but hopefully you land on an idea that resonates with the market and you then switch from this sort of innovative search into kind of, hey, we need to execute on what's working. And the challenge is that the sort of tools and processes and even people that you need in that first phase can be different. Well, are very different from the people that you need in that execution and growth phase. So it's a complex problem, but I would say with Looker as an example, Looker was a very, very, very engineering driven company and highly innovative along that dimension as it tried to figure out the right product market fit. And I remember the cto, Lloyd, I'm sure you know, Lloyd Tapp, that guy's amazing. He's amazing. Yeah. I remember on the interview that I had with him, it was interesting cause he said two things. One was that the company was at the time struggling to put together a roadmap. And then the other thing I remember him saying was he has a personal ethos, that the engineer has the pen. I wondered if he ever put those two things together. So one thing that you have to think about is how do you bring engineering along into a world where they're going to have a lot of say in the day to day about the product roadmap, to a world where maybe sales and markets are going to start to have more influence. Because once you have that product market fit, the reality is you scale by listening increasingly to your customers, figuring out how to synthesize those roadmaps in a way that provides predictability as you try and scale the organization. So that was sort of the challenge that we faced early on there.

And that's a pretty big transition right from when the engineering, I mean, when you were talking about Lloyd having the pen, what you mean is that the engineers basically rule all of it, right? And then what ends up happening is Sales team starts selling and customer success team starts engaging with different customers and you get all this feedback that comes back from customers and product and Eng might have a particular view and sales and marketing have a different view because they're trying to close that big customer and want one particular feature.

That's it. I mean, it's a matter of the levers of control shifting a little bit. And it's healthy to have this tension, I think. But you do from a, from this dimension of the problem, you do have to win over engineers. You have to figure out how to empathize with them.

How do you do that?

Well, at Looker, fortunately we had a really healthy culture to start with. So before I get into what we did there, it always helps to make sure that you're building on top of an environment that's collaborative, open to feedback, et cetera. And walking in the Looker. I was extraordinarily fortunate there. I think where we had to make some changes though was that the engineering department was not necessarily connected in terms of communication with the go to market functions. That is to say that the sales and marketing folks were asking for better roadmaps and more predictability and this sort of thing. And the dots weren't being connected between the day to day operations engineering and what the go to market functions needed. And you know, there's some challenges all around there. I think with engineers though, the way that you win them over is you have to help them understand that in giving up this sort of day to day control, what are they going to get in return? Right. Like an engineer's worst nightmare, at least when I'm in that capacity is that like they're being directly told by salespeople what to build every single day and they lose the freedom to innovate. They lose the freedom to go back and address like core technical debt concerns and things like that. In return for giving up some of that freedom. You want to show that now that you've got product market fit and you can scale, you can have much bigger impact in the world. So what we did at Looker on this dimension was first, the company had never really sat down with engineering and some parts of the product organization well and just expressed the potential that the business had. Like sit down, let's walk through the Gartner Magic quadrant or external validation from other sources. Just talk about the position in the world, the time to switch from innovation to execution. Because we have figured out something that is working. Like engineers have to be brought into the strategy in that way they can't just be excluded, locked in the basement, throw pizza downstairs and hope the code comes out. You gotta bring them along. The second thing that we did there, because our engineering team had particularly strong opinions about what should go into the product, we started to loop in customers and have them talk at all hands. We also happened to have an NPS database that was going unused with some great raw feedback. So I started to get the team to synthesize that information and present it directly to the engineering team. And again, this is providing more signal to engineers about hey look like here's some other things to consider that you might not have seen. You weren't seeing all of the picture before for so here's some other signal. And then the final thing is as you build your roadmap, you have to I believe, take into consideration technical debt and concerns from engineering. This is the thing I see go wrong at companies of every scale that like the balance between tech debt and feature work is something that's always hard to get right because you invest in different levels depending on where your company is in the product lifecycle. If you're a brand new company and you really just need to innovate, it's fairly safe to assume that you're going to throw away most of your code. When you are transitioning from that innovation to execution phase though, you're probably going to keep a little bit of it. And then when you're full on executing and need to maintain, you need to be having a consistent way that you address ongoing technical debt. So here what I wanted to do at Looker was make sure that the many projects that were kind of in the backlog related to building up the technical systems were considered right alongside some of the project work. And ideally, like my favorite way to solve this problem is if you can kill two birds with one stone, like design your new product work so that it actually pulls technical improvements along with it. And at Looker we were able to do that in multiple areas. I was really proud of the work the team did in up leveling almost all of our systems. We went just to give an example, just to toot the horn there we went from I want to say on prem but a not necessarily cloud first approach to a fully multi cloud solution within about a year and a half. And that required a lot of new feature work alongside technical debt, security improvements, et cetera. And it was good to see the team rallying around that. It required them understanding the strategy though, so I'll stop there. Tomas, I don't know if you also.

Want to oh, great. As you were talking, one of the things I was thinking about is the number of startups that I've seen that when they start to get into this transition and the sales team starts banging on the product, the number of times people talk about, hey, we need to pause all product development to fix our technical debt and how big of a strategic bet that is for the company. Can you talk about your perspective on that?

I think I've heard that in many cases and sometimes you're in situations where it's like, hey, we got a burning platform, there's really no way forward unless we do a complete rewrite or something like that. When I was at Power Bi a long time ago, I remember our mobile application was built on Windows Mobile. Hey, we want to be competitive in the market. Probably not the right way to do it. We had to stop all development, throw that thing out and like just do a complete overall, like. So there are situations where that can happen. But I would claim that it's much better if you can walk and chew gum at the same time. That if you build the right, if you build the right layers of collaboration between what the sales and go to market functions want, the PM organization as well as the engineering team, it is totally possible to figure out a balance that allows you to address technical debt as you go along. And again, I think even going further than that, I try to avoid even calling it tech debt where I can. I like to plug it into new feature work and just say, hey, like, we're rolling out this new feature. Why don't we, alongside of rolling out this new feature, think about like whether or not we might use Kubernetes as the backing architecture for this like, and pull through some technical improvements alongside so it doesn't look, you know, tech debt has this connotation of being drudge work. It doesn't have to be drudge work. It can be work that actually improves the overall product experience. And that's the way I like to think about it. You know, when you're fixing tech debt, you're kind of fixing the foundations of your house. You're improving the ability to build all products. And I think that's the way I look at it needs to be celebrated more than it traditionally is.

That's true. The framing is so important. I mean, there's nothing, I used to be an engineer. There's nothing worse than an engineer being given somebody else's old code. And to say, hey, your responsibility is to fix and maintain this. It takes all the fun out of the problem.

Solving, you know, yeah, just rewrite this old.

It will do exactly the same thing. It'll just work better. No one will notice.

So I think the other thing there is, you know, we talked primarily about engineering. I think it goes both ways. Those, the go to market functions as well have to change in that world. I think the critical thing there is road mapping. And as you switch from that innovation stage where you don't quite know what you're going to do necessarily from quarter to quarter to a mode where you do understand the go to market, the ability to have roadmaps that are consistent solves so many problems. They're kind of the fundamental unit you need to keep engineering and sales in sync. But they also help with communicating vision and what to expect to your customers, particularly in the enterprise. I think where is where this matters.

How did you do that at Looker? I mean we didn't have a roadmap and then, you know, we needed to create a roadmap that's not, you know, Rome wasn't built in a day. A roadmap wasn't written in a day either.

You know, I give, I will, I want to, before I explain this, give all credit to the team who actually implemented these changes. So Sandra Orozco out there might be listening to this and she's one of the most phenomenal TPMs I've ever worked with. But at Looker, yeah, we, when I walked in, we didn't have a roadmap. We didn't have what I believe is fundamentally necessary to build a good roadmap, which is tools like JIRA that engineering managers and PMs use on a day to day or week to week basis to track the work that is going on. Like that is to say a roadmap comes in two parts. One is like vision. It's easy to build a visionary roadmap, right? Like you ever hear like Elon Musk talk about like landing people on the.

Four sentences for Tesla or you know.

It'S like okay, yeah, exactly, someday we'll get the full self driving whatever. But it's easy to talk about the long, long term roadmap. What is more challenging though is like hey, quarter out. Like tell me what you're really going to land and can you do it consistently at a 90% confidence rate. And in order to do that you have to have a certain culture of engineering and product and you need tools to support that culture. So the tools that we implemented at Looker, again with the help of an entire team, were to bring in Jira as our work management tool to bring in an engineering culture that was heavily weighted toward predictability and quality. I told the team continually, like, I will give you all the freedom and control in the world as long as you can be, do it predictably. So that sort of culture came into play. And then delivering consistently and over communicating our roadmaps to our stakeholder teams so that they could be brought in and not be surprised about what was coming down the pipeline. And in order to do this, I sort of put the team through a bit of a torture chamber. So I started in October and we rolled out a small version of this process. Jira. I gave them a whole strategy for how to build a whole process for how to build product strategies called dibs. I stole that from Spotify, if you want to look it up. And we ran through a six month product planning cycle in two weeks in October. And I told them, hey guys, good job on the practice round, we're going to do this one more time. So we did it back to back. We did two six month planning processes back to back in order to kick off the year. And I got a lot of complaints and pushback for that. Why are we doing all this unnecessary paperwork and extra bureaucracy? But having done that twice, we landed in the new year with the team just able to execute, understanding the culture we were trying to head toward. And then I was able to kind of pull back. There's nothing better than being in a leadership position where you've, you've just trained everyone else to do all the work. So I was able to start like pulling, pulling back after we got that sort of discipline tooling and so forth in place.

Nick, we got, just on that point, we got a question from the audience. Do you tie new features in tech debt work to annual or quarterly OKRs? Do you tie anything in the roadmap to OKRs?

Yeah, yeah, absolutely. I don't know how much we're going to get into to okrs. I have a lot of opinions on that. But yeah, absolutely. I think philosophically I like to have an 85% accurate prediction of what my organization is going to build within a quarter. And in order to do that, you know, you can use okrs, you can use many constructs. Like I think OKRS is depending on where you work, one of the more popular ones. But for me, every quarter, and that doesn't mean, that doesn't mean stuff things into a quarter, what I ask teams to produce is a slightly ambitious plan. Right. Given the resources that you have, tell me how you would want to deploy them. Give me a slightly ambitious plan that you feel you can hit with 85% confidence in three months. If something you're proposing takes longer than three months, we should have a discussion about how to break it down into something that's a little bit more predictable. And then ideally you what? Just one anti pattern I see is people say, oh yeah, three months, that sounds great. But then every single project that they propose takes exactly three months. So this is also an anti pattern just for ems out in the audience who might be listening to this. If your estimates always land on like a month boundary or a quarterly boundary, like an experienced director will see that and assume that you've kind of whiffed it. So do enough estimation to give a high confidence estimate with an 85% accuracy and that will most likely not land on exactly a quarterly boundary.

That's awesome. Okay, so let's keep going with the theme. So we talked about that transition from product market fit innovation to execution. You've now gotten the engineering team into a place where they are bought in and excited about involving sales in the product roadmap. You've got a product roadmap. Now you've got to scale out the team. And one of the things that I really liked about your writing is you have strong views on the difference between a manager and a leader. Can you go a bit more into detail about the differences between a manager and a leader?

Yeah, I think. Look, it's not, by the way, whenever I answer this question, I think people interpret it the wrong way. Look, managers can be leaders and ideally you get managers who are leaders. But I also want to contrast that by saying the expectations for leadership are not quite the same as what you would expect for management. Like with managers. And we were just talking about it. What I really want is someone who can take care of their team, take care of the people on the team. All right, one, that's my high order bit for managers. And then two, they need to be able to execute against a strategy or a roadmap with predictability and quality. So management is all about building things and getting things done. And ideally a sort of stable, predictable, low risk fashion. Right? That's what a manager does. Now think about your favorite leader and, and what they do. Right? Pick anyone and if they're stable, they're probably not doing what most leaders are brought in to do. Leaders are often brought in to facilitate change, like to set new directions, think about the long term horizon. They use empathy and charm and the heart to inspire people to take on new challenges. So they're really, if anything, you know, we talked about managers being forces of stability and predictability. You can think about leaders in the opposite fashion. You want leaders who are going to come in and inspire people to do something different. And they don't necessarily have to. Leaders in order to get what they want done, they don't necessarily have to have people recording into them. You think about like, managers might have support subordinates or direct reports, but anyone can be a leader as long as they can get followers, people who believe in their ideas and the vision that they're trying to project. And I think that's the fundamental difference. As I would describe it, I think leaders can come from anywhere. And I try. And in any organization I build, construct systems that we can go find them. And the reason I believe that's so important is because as organizations scale, the number of people obviously scales linearly. The number of managers does not scale linearly, it scales logarithmically. So if you're not tapping into every single potential leader in your organization in a systemic way, you're missing out on opportunities to find talent, change agents, et cetera. Okay, I'll start ranting about this because.

No, no, no, this is awesome. This is awesome. So how do you. So as you talk about that phase, one of the key things we focus on at a board level when working with startups is the middle management becomes super important as you start this transition of actually growing. And those managers are critical because you need help recruiting, you need help managing. Like you said, the number of employees in a company is going to grow by 3x 4x whatever it is in a given year. And so you can't just have a level of VPs and then a bunch of individual contributors and expect success. Right? You need, you need a bunch of. And within the engineering teams, you know, there's lots of different ways to construct it. You could bring it, you could bring experienced people in house, you could promote from within. One of the things that you talk a lot about is finding out, you know, leadership opportunities within a business. How do you think about building out that? And you did it at Reddit within 18 months, which is just incredible. You know, how do you do it and how did you grow that middle management layer to be successful?

Yeah, I mean, for folks who are listening in this, the story about Reddit was about scaling from a startup scale. We had about like 35 people. And then we, I think we ended, we had like 170 or something like that. And we did it really, really Fast. So I see this pattern happen at other startups who are experiencing hyper. Hyperscale. And the way I like to think about this is two parts. First, you know, there's a whole discussion we could go into about organizational design. I won't go down that long rat hole. But you do have to think about how you want your organization to be structured. The way I like to structure teams at scale is build on a strong foundation of shared services and shared capabilities. Think about your product silos and who's going to lead those product silos. And then on top of all of this, think about teams that help you better understand your customer, like data science or growth teams. So that, roughly, is the shape that I'm aiming for. You might have a different shape. The key thing is, once you have that shape, you're going to recognize that you need to bring in a certain number of managers to. To fill key roles, and you should be able to identify that really quickly. And then you got to look at your army. Who do you got working for you? You gotta. You gotta ask yourself, do I have the people on staff that I'm gonna invest in to take on management roles, or should some of these people stay in the IC track? Do I need to go external? And at Reddit, we had that problem really quickly. Cause the day I walked in, Steve, the CEO, was like, nick, we gotta scale up really, really fast. So within the first couple of weeks, I was starting to think about how do we suss out the difference between future managers and people who want to go down that IC track? And I came up with this sort of test, call it like the. Oh, you got a slide for this?

Yeah, I have a slide. I forgot about the slides. All right, but here's a slide. And here. Oh, that's way too big. Okay, let's see.

Oh, it's all good. I'll talk through it. It's. It's sort of a test. If you've ever seen the movie Blade Runner, they've got this thing called the Voigtkampf test. And the idea is you're sitting across from someone who looks like a human, but you're not quite sure. You got to figure out if they're a human or a robot. And the Voigtkampf test is a series of questions that the Blade Runner asks the prospective candidate. If they pass, great. If not, I guess the Blade Runner shoots him. And you can take that same sort of approach to sussing out potential managers into people who will become engineering managers or people who should maybe go down an IC or architect track. And we, we did this at Reddit, by the way. The way that this was structured was not a formal test per se. This was something that, you know, came up in the course of one on ones with people on the team. All right, so we didn't.

So you weren't walking people into a confessional and asking, you know, behind a closed curtain.

Yeah, exactly. But I mean, I think it's important at early stage startups to start putting this framing in place that you can have a management track and a purely technical track when you get later on into larger stage companies. I see companies that haven't quite solved this and they're still struggling with it. And the way that manifests is you might have a lot of people with the tech lead title. I've consulted at a lot of different companies, I've worked at a lot of different places. Every single one of them had a different definition for tech lead. And for me, when I'm trying to build out an organization, I like to clarify that like you can have tech leads at certain scales of your organization, but as you climb through the ranks, I like that to go into a dedicated IC track or a dedicated management track. So these are some of the questions roughly that we asked. Do you care more about people or technology? For me, again, if you're going to be a manager, people has to be the high order bit thoughts about shipping toward a deadline. When I talk to, in general, when I talk to architects, they're not as focused on deadlines. They're thinking about the longer term implications of the technology choices. That's what gets them excited. PM goes to two of your direct reports and asks them to immediately start working on something. If you're an engineering manager and you don't immediately shut that down, I would have some concerns. But you do have, there is a certain type of engineer who will be like, oh yeah, let's just jump right into that and go figure out, start, start problem solving. Right. And the other thing, you know, if you're, if you're going to be a management role, particularly you know, in a hyperscale startup, you end up spending a lot of your time staffing and you got to build up a thick skin by getting LinkedIn rejection. So I threw, for Reddit's purposes, I threw that, that in. I think, you know, as you, as you think about this for your own companies, you're going to come up with a different set of questions. The big takeaway from this, do put in place as early as you can a Way to think about these two different tracks, because if you don't, you will eventually be asked about it. You're going to have to solve this problem, and I think it can be solved fairly early in the stage of a startup's life. Awesome.

Touching on that last point. One of the stories that I really appreciated in your writings was talking about the interview you had at Microsoft when you were recruiting a machine learning engineer and you met a machine learning engineer who was very quiet, very technically sophisticated, but very quiet. And then you were part of the group that votes on who's admitted to hiring group, and you were the only no vote. Can you walk us through that story and what the lesson was there?

Yeah. So this was a. I guess I'll share all my mistakes. We'll just dig into my trove of bad decisions. Thank you, Tomas. Yeah, no. So I think over time, my opinion on what it takes to be a leader has changed. And I remember this incident well because it was a moment where my manager came in and corrected me in my way of thinking. So I had a candidate come in who was interviewing for a machine learning technical role, and he passed all of the. Of the interviews questions. Well, all the technical questions just knocked him out of the park. I remember vividly because I used to pride myself, this is the old Microsoft days, on having impenetrable questions, dynamic programming. I would never do this today, but like, back then, I guess it was more common. And this dude just came in and knocked them all out of the park. And the challenge is it was just like, really quiet. His name was Tony, by the way. So Tony was really just quiet. Like, he went through the questions, but there was no charisma. And I, when we got to the interview panel, you know, went around the room and everyone gave higher and higher, higher, higher. And it got to me. And I was like, no, I don't see Tony as a leader, because I can't imagine how someone this quiet and reserved would be able to come in and lead. Like, how is someone this introverted going to lead? And so I said no. And then my manager pulled me aside, and I appreciate him now for having done this. I didn't understand it in the moment, but now I definitely appreciate it. He said, nick, I'm very disappointed in you.

Really? That's how he started.

Well, this is millennial speak for you're an idiot. So I appreciate him phrasing it in that way as opposed to what I'm sure was going through his head, which is, Nick, stop being so dumb here. So he Was like, Nick, like really disappointed in you. Why don't we, why don't, why don't you just trust me? We'll give this guy a hire and we'll see how things turn out. And Tony ended up being one of the best hires I ever met. I worked with that guy for like another eight years. When, when we. He brought him into the team, he started mentoring people. I mean, he, I mean he was stunning. Like I was completely wrong about him. And he had a way of leading that, yes, was quiet and reserved, but was also incredibly impactful. So it just taught me that like, leadership can come in many, many styles. And that was the first moment where it really hit me that you can't. There's no defined way to lead. I push people on this all the time. If you look behind me, you'll see I've got a bunch of leadership books on my shelf. And what I tell people is you can read all of these things, but there is no formula. Like, leadership comes in so many shapes and styles. It's all dependent on the situation you're in, bringing your own personal strengths to the table and understanding how you can be the person to step up in the right moment. And so Tony was the right person. He had the knowledge, skills and abilities and machine learning that allowed our team to thrive and get to the next level. And it's just not something that came across during the interview. So I'm much more open minded about those things now.

Is that something that you think you now can tell with an interview, even if somebody is quiet, they've got the right leadership style? How do you sort of suss that out when you're meeting new folks?

Yeah, I think what I've since come to believe is you want to look at how far people have come, what. How far people have come and what they've overcome, you know, is a good indicator. And that may be different for different classes of folks. I won't get into that at this exact moment, but I think that I increasingly look at a combination of track record and whether or not that person has unique passion, skills or abilities that would help them lead in the moment. And it's hard to describe, I think like whenever I do interview panels, I like to make sure I get a lot of different viewpoints on the candidate. But spread over a number of people, you can start to suss out whether the candidate is particularly excited or insightful or inspired by the role that you have. I don't know if that necessarily, I would say is a requirement, but in terms of spotting potential leaders. One of the things I look for is that element of passion, that spark. And then when I'm trying to generate new leaders, I try and think of ways to give people that spark. Right. So not everyone has that at every moment in their career. Right. Not everyone at all times is working on the exact one thing that is going to spark their passion. But I think as a leader, you have to continually be figuring out ways to give people that spark. So I'll stop there.

Yeah. And you call it the blue flame, right? And the blue flame, right. What's, what's the blue flame?

That, that is that. Man, you've done a lot of research. So. Yeah, so the blue flame, I think is. It's when you get the intersection between what someone's personal passion and motivation are like the thing that like they just love working on. When you get that intersected with what the business needs at a given moment, you end up in the blue flame, the hottest part of the flame. And as a leader, I'm always trying to think through in one on one conversations or career coaching conversations with people who work for me. I have this in the back of my mind. Is this person in the blue flame or are they, or could they be, should they be encouraged to do something else that would get them there? Right. So you can figure that out by. It's not again, like one on ones are sort of the magical element for, you know, for anything you might do as a manager. You can, you can generally suss out the best of it in a one on one just understanding what really inspires people. But you know, the way I like to suss this out is to talk to people about their longer term goals and where they see themselves. And I always keep that in the back of my mind when we're talking about product roadmapping and so forth and so on. And if I see an opportunity that someone's long term ambitions and where they want to go intersect with what the business needs to take a bet on, then I go and try and give people that chance and see where they end up. And so far it's worked out really great. I mean like if you get people on something they're passionate about, you get a little bit of extra horsepower. And then also I think it builds a better connection between me and them as a manager. Like, people want to know that you're thinking about them and have their best interests at heart as you build the organization. That's the, it's the easiest thing to lose when you're getting to Big scale. I'm terrified of being in a world where I manage hundreds and hundreds of people but lose the ability to really understand and think about what makes people passionate. I felt if I lost that, I would lose myself as a leader.

Yeah, that's wonderful. Awesome. One more question from me and then we've got loads of questions from the audience. So. So just to kind of wrap things up, you do a lot with an organization called Dev Color, which is, you know, the mission is around diversity and inclusion engineering. It'd be great if you'd spend just a couple of minutes talking about the organization, why you're so excited about it.

Yeah, yeah. So just guys, this, if you, if you don't know about this organization, like I'm going to talk about it, but go to the website right now so I'll talk about how I got introduced to this organization. I had been in Seattle for a very long time in my career. I spent about 15 years in Seattle. And not to knock folks who are from there, but it's not a particularly diverse part of the world. And I, as a black software engineer, I spent most of my career.

All.

Of my career at Microsoft without seeing another black VP of engineering or someone in a high level role. So I was, I was fairly frustrated by this. And when I moved to San Francisco, one of the first things I did was I attended a meeting of deaf color, 300 other black engineers in the same room. It was absolutely mind blowing for me. Like first time in my career I had ever experienced this. And from all career stages to other black execs, people mid career trying to rise up and then people fresh out of boot camps, all, all of the above and one place. So I joined the organization. I got so passionate about it that they later on invited me to be on the board. With my first nonprofit board, I discovered that you have to, you have to pay them to be on the board for the privilege. Yeah, well, it is what it is. Anyway, so I'm now on the on the board and I encourage you to look into this organization. It is an organization that uses a squad model or lean in circle model to help teams of black software engineers achieve extremely ambitious goals. So when I was doing my deaf caller squad, I set my goals. I'm not making this up by the way. I set my goals as doing more public speaking. I wanted to do more writing and I wanted to get more connected with the venture capital community. Today a few years later, I have done speaking events literally around the world. My blog posts and writing, which Tomas has referred to here have gotten hundreds of thousands of views. I'm an angel investor in more than 30 companies and I've achieved all of those goals and I track that directly back to my experience at Dev College. So shout out to them, please go look at this. If your company is interested in finding ways to support diverse engineers in your team, like this is the best organization, like period, end of story. Like go talk to them right now. Okay, I'll stop. Awesome.

Thank you so much. Okay, going through some of the questions from the audience. There's a real quick one. What does ICTrack stand for?

Oh, sorry guys. So individual contributor. That's. Sorry, really tactical question there. So you think about if you're an individual contributor, you know, how do you, as your career goes on, how do you continue to contribute if you don't necessarily want to be in management? So a lot of people think like, oh, if I want to climb the ranks, I have to be in management. That's not true. Like a well designed. If your company has a well designed, well thought through career model, they'll have a ladder for people who want to go into management, which is a very, very different skill set. Trust me.

Right.

You definitely don't want your head of your management career track messing around with checking things in. Trust me, at large companies won't get into that. And then you've got your IC track where you might move up and become an architect or a distinguished engineer. But in that role you're going to stay a little bit closer to the day to day of the code and, and technical decisions and things like that.

Awesome. A related question, how does the relationship between an architect and an engineering manager work?

So I have an opinion on it. This works differently at every company. But I think about what engineering managers or VPs of engineering or directors of engineering are doing. They're really managing the people and the resources and trying to make sure that the people are aligned to best contribute to the company goals. Whereas your architects are like diving in and solving specific technical problems or setting long term technical strategy. So the way that I like to design organizations is that you know, every EM Director or VP of engineering is paired up with the equivalent IC technical track leader. So for me as VP of engineering, I'd want to be paired up with like a distinguished engineer or high level principal IC to just make sure that I understand what the hell I'm talking about and that recurses down through the organization. So I want to see directors paired up with. It could be our. It depends how you title these people at your company, but might be an IC staff level architect or something like that, all the way down to the EM level if possible. That's sort of an idealized state. I don't think I've ever worked at any company that was able to do that one to one mapping. But it's what I try to aspire to because it makes every pair gets. If you do that, the pairs get stronger. Like your, your architects have a better understanding of the limitations associated with the org structure and the people your ems get. Don't lose the sight of the technical challenges. It's, it's a good team, it's a, it's a good unit to bring together to make sure that the right decisions are being made. Awesome.

Great answer. Okay, going back to that, we had a bunch of questions around Roadmap. How do you measure and communicate predictability for feature delivery?

So I can tell you what not to do and I can. So let me. Actually I'll start with what to do and then I'll tell you what not to do. Because as an engineer there's a level of tracking that I can't tolerate. You guys have to remember that I came from an engineering background even though I've been like CPO and done all this other stuff. The easiest way to do this is if you can get project sizes down to what I was talking about earlier, something less than a quarter. If you can agree across the board that you're going to whittle down whatever you agree to do down to less than quarter size chunks and you're going to track those in some consistent way. Like the way I like to do this is, you know, we have engineer engineering managers and directors. On a week to week basis they're, they're going to look at the project status for the things they've committed to for that quarter and say whether or not they're on or off track. And if you agree with that premise, then it becomes fairly easy to understand whether or not your predictability is accurate or not. You can, a quarter goes by, you can look at the projects that you agreed to to ship and did you ship them or not? And if, if that number is less than 85%, you might go back and, and adjust and so on. There's more detailed ways to do this that I will leave it to you as the listener to describe if you want to use them. I personally in life am allergic to story points, which are another way to do this. If you really like a fan of Agile methodology, you can go read up on story points another way to do this, which I saw implemented at Microsoft. You can go a level further down. So I was talking about, hey, teams should commit to projects. You can go even further down and get individuals to commit to specific okrs and specific project line items. And I think when you, in my opinion, when you start to get to that level of granularity, it breaks up team cohesion a little bit. So I don't recommend you using this sort of tracking methodology all the way down to the level of a specific individual. I like to keep it at the level of the team so the team is motivated to get projects done. I think you get more positive behavior if you measure in that way. Awesome.

Great answer. Okay, last question. Because I know you're a very busy guy, we got to get you back to running the engineering part of Twitter. How would you recommend a very early stage startup interview a potential CTO or VP Eng?

So it depends how early stage. Does it say anything else about that in the question?

It says so here's a question from Ben. How would you recommend a very early startup find good technical talent, potentially a cto? How do you vet these candidates if you are a non technical technical founder?

Okay, got it. So if you're a non technical founder at a very, very early stage startup trying to assess the skill set of a CTO or VP of Eng, I recommend you use your external network to bring in people on the interview panel. So I would go to the board and I would ask them for referrals if you, if you do that in conjunction with stakeholders who are currently at the company. Company. So the implication of your question is that you may not have the necessary people in your organization to evaluate whether or not the VP of ENG is going to have the strategic and people management capabilities or whether or not the CTO is going to have the technical depth. If you don't have that skill set in the building, don't be shy about asking for help from your network. The catch is though, you do need to have, you do need to have stakeholders from within your company be a part of that decision. I remember when I was interviewing for Reddit, getting the opportunity for me to meet the existing engineering managers and tech leads was a very valuable sort of conversation. I think they got a sense of what my style would be like when I was going to come into the organization. At the same time I sort of tell they didn't, they didn't really know what a VP of ENG was supposed to do. So I don't know if they, I use that as an opportunity to explain to them, you know, so many of the things we talked about here, like, you know, what does it mean to be to build an organization and kind of get them comfortable with what I was intending to do. But in terms of that internal group of people evaluating me, I don't know if they necessarily weighed in as much. I remember I think Steve brought in a guy named I like to do it quite a bit like Marty Weiner to help evaluate my skills as a VP of Eng for that particular role. So don't be shy to ask external is all I'm saying.

That's great advice, Nick. Thank you. This has been incredible. I've learned a ton about you and really grateful for sharing all of your insights throughout your incredible career. Thank you so much for the time.

I really appreciate the invite and the chance to lose my voice with you. And if I guess if folks want to connect with me, you can find me on Twitter, which is awesome product and awesome place to work. And you can connect with me Ickcald on Twitter.

Awesome. Thank you so much, Nick. Thanks everybody for the questions.