Growing Product & Engineering Orgs with Twitter VP Engineering + Redpoint Ventures Managing Director

Okay, and you can begin countdown now.

Welcome everybody. My name is Tamash Tinguz. I'm a managing director at Redpoint and I'm thrilled to be joined today by Nick Caldwell, a friend who is currently VP of Engineering at Twitter, and we are going to be talking about Growing product and engineering team organizations from zero to IPO. I'm thrilled that Nick is with us because he's got an incredible background. Nick grew up in Maryland. His first computer was a Tandy 1000. I had a lot of fun researching you, Nick, by the way. Uh, and you set up your own BBS on that machine. Then you went to a magnet school, you rose up through the ranks of MIT, and you started working at Microsoft, and your first car was a Nissan 350Z. What color was it?

Oh, it was silver, man. That was amazing. I bought that car before I even got my first paycheck. I'm not lying.

Is that true?

Wow. Yeah, I was making bad decisions back then.

Then you rose up through the ranks at Microsoft. You started the Power BI team, grew that to 300 people. Then you switched from B2B to B2C, which is rare to see. You became the VP of Reddit, which is the front page of the internet, and you tripled the engineering team there in 18 months. Then we got to know each other because you became the Chief Product Officer at Looker, which then afterwards became, was acquired by Google, and now you're VP Eng at Twitter. So you've seen both B2B and B2C, you've seen incredible scale, you've got a wonderful perspective on all this, and I also learned that you love video games and sneakers on the side, and it's clear you're a LEGO fanatic from your background too.

Oh yeah, I had a lot of free time between my current job and the last, so we put the Millennium Falcon together. But thanks for the intro, Tomáš. I was looking forward to to this fireside chat for quite some time, and I really appreciate you inviting me to the stage here.

No, thrilled to be having the conversation. So let's jump into the question. So one of the things that I really wanted to talk to you about was when founders are starting companies, the first goal is really the establishment of product-market fit, making sure that customers actually want the product that you're building. And then afterwards, there's this transition from innovation to execution. So now that you've built a product, you actually want to start scaling the org. You've made that turn several times. The big change that you've written about in the blogs is sales and marketing now starts to influence the product roadmap. What does that mean? What does it feel like in an org? How should you, as a leader within a startup, sort of anticipate and expect— what changes should you expect in the org when you make that transition?

Yeah, you see a lot of companies struggle with this. And I think that that blog was written in the context of Looker, which is a company I in part came to help this specific problem. But, you know, early on in your product lifecycle, you're obviously in an innovation phase. You're in that sort of random walk trying to find product-market fit as quickly as possible before your funding runs out and you have to get a different job. But hopefully you land on an idea that resonates with the market, and you then switch from this sort of innovative search into kind of, hey, we need to execute on what's working. And the challenge is that the sort of tools and processes and even people that you need in that first phase can be different— well, are very different from the people that you need in that execution and growth phase. So It's a complex problem, but I would say, you know, with Looker as an example, Looker was a very, very, very engineering-driven company and highly innovative along that dimension as it tried to figure out the right product-market fit. And I remember the CTO Lloyd, I'm sure you know Lloyd Tabb, that guy's amazing. He's amazing. Yeah, I remember on the interview that I had with him, it was interesting 'cause he said two things. One was that the company was at the time struggling to put together a roadmap, and then dot, dot, dot. The other thing I remember him saying was he has a personal ethos that the engineer has the pen. I wondered if he ever put those two things together. So one thing that you have to think about is how do you bring engineering along into a world where, you know, they're going to have a lot of say in the day-to-day about the product roadmap, to a world where maybe sales and marketing are going to start to have more influence. Because once you have that product-market fit, the reality is you scale by listening increasingly to your customers, figuring out how to synthesize those roadmaps in a way that provides predictability as you try and scale the organization. So that was sort of the challenge that we faced early on there.

And that's a pretty big transition, right? From when the engineering— I mean, when you were talking about Lloyd having the pen, what you mean is that the engineers basically rule All of it, right? And then what ends up happening is sales teams start selling and customer success teams start engaging with different customers and you get all this feedback that comes back from customers and product and eng might have a particular view and sales and marketing have a different view because they're trying to close that big customer and want one particular feature.

That's it. I mean, it's a matter of the levers of control shifting a little bit and it's healthy to have this tension, I think, but you do from a from this dimension of the problem, you do have to win over engineers. You have to figure out how to empathize with them. How do you do that? Well, at Looker, fortunately, we had a really healthy culture to start with. So, you know, before I get into what we did there, it always helps to make sure that you're building on top of an environment that's collaborative, open to feedback, et cetera. And walking into Looker, I was extraordinarily fortunate. There. I think where we had to, to make some changes though was that the engineering department was not necessarily connected in, in terms of communication with the go-to-market functions. That is to say that the sales and marketing folks were asking for better roadmaps and more predictability and this sort of thing. And the dots weren't being connected between the day-to-day operations and engineering and what the go-to-market functions needed. And, you know, there's, there's some challenges all around there. I think with, with, with engineers, though, the way that you win them over is you have to help them understand that in giving up this sort of day-to-day control, what are they going to get in return, right? Like an engineer's worst nightmare, at least when I'm in that capacity, is that like they're being directly told by salespeople what to build every single day, and they lose the freedom to innovate, they lose the freedom to go back and address like core technical debt concerns and things like that. So In return for giving up some of that freedom, you wanna show that now that you've got product-market fit and you can scale, you can have much bigger impact in the world. So what we did at Looker on this dimension was first, the company had never really sat down with engineering and some parts of the product organization well and just expressed the potential that the business had. Like, you know, sit down, let's walk through the Gartner Magic Quadrant or external validation from other sources and just talk about the position in the world, the time to switch from innovation to execution because we have figured out something that is working. Like engineers have to be brought into the strategy in that way. They can't just be excluded, locked in the basement, throw pizza downstairs and hope the code comes out.

They've gotta— [Speaker] Yeah.

[Speaker] You gotta bring them along. The second thing that we did there, because our engineering team had particularly strong opinions about what should go into the product, we started to loop in customers and have them talk at all hands. We also happened to have an NPS database that was going unused with some great raw feedback. So I started to get the team to synthesize that information and present it directly to the engineering team. And again, this is providing more signal to engineers about, hey, look, like, here's some other things to consider that you might not have seen. You know, you weren't seeing all of the picture before, so here's some other signal. And then the final thing is, as you build your roadmap, You have to, I believe, take into consideration, you know, technical debt and concerns from engineering. This is the thing I see go wrong at companies of every scale, that like the balance between tech debt and feature work is something that's always hard to get right because you invest in different levels depending on where your company is in the product lifecycle. If you're a brand new company and you really just need to innovate, it's fairly safe to assume that you're going to throw away most of your code. When you are transitioning from that innovation execution phase, though, you're probably going to keep a little bit of it. And then when you're full on executing and need to maintain, you need to be having a consistent way that you address ongoing technical debt. So here, what I wanted to do with Looker was make sure that the many projects that were kind of in the backlog related to building up the technical systems were considered right alongside some of the project work. And ideally, like my favorite way to solve this problem is if you can kill two birds with one stone, like design your new product work so that it actually pulls technical improvements along with it. And at Looker, we were able to do that in multiple areas. I was really proud of the work the team did in upleveling almost all of our systems. We went, just to give an example, just to toot the horn there, we went from, from a, I want to say on-prem, but a not necessarily cloud-first approach to a fully, you know, multi-cloud solution within about a year and a half. And that required a lot of new feature work alongside technical debt, you know, security improvements, et cetera. And it was good to see the team rallying around that. It required them understanding the strategy though. So I'll stop there, Tomasz. I don't know if you also want to— No, that's great.

I mean, you know, as you were talking, one of the things I was thinking about is that the number the startups that I've seen that, you know, when they start to get into this transition and the sales team starts banging on the product, the number of times people talk about, hey, we need to pause all product development to fix our technical debt, and how big of a strategic bet that is for the company. Can you talk about your perspective on that?

I think I've heard that in many cases, and You know, sometimes you're in situations where it's like, hey, we got a burning platform. There's really no way forward unless we do a complete rewrite or something like that. I, when I was at Power BI a long time ago, I remember our mobile application was built on Windows Mobile. And, you know, hey, we want to be competitive in the market. Probably not the right way to do it. So we had to stop all development, throw that thing out, and like just do a complete overhaul. Like, so there are situations where that can happen, but I would claim that it's much better if you can walk and chew gum at the same time. That if you build the right— if you build the right layers of collaboration between what the sales and go-to-market functions want, the PM organization, as well as the engineering team, it is totally possible to figure out a balance that allows you to address technical debt as you go along. And again, I think even going further than that, I try to avoid even calling it tech debt where I can. I like to plug it into new feature work and just say, hey, like, we're rolling out this new feature. Why don't we, alongside of rolling out this new feature, think about whether or not we might use Kubernetes as the backing architecture for this? And pull through some technical improvements alongside so it doesn't look Tech debt has this connotation of being drudge work. It doesn't have to be drudge work. It can be work that actually improves the overall product experience, and that's the way I like to think about it. When you're fixing tech debt, you're kind of fixing the foundations of your house. You're improving the ability to build all products, and I think that's the way I look at it. It needs to be celebrated more than it traditionally is.

That's true. The framing is so important. I mean, there's nothing— I used to be an engineer, and there's nothing worse than an engineer being given somebody else's old code and just say, hey, your responsibility is to fix and maintain this. It takes all the fun out of the problem solving, you know?

Yeah, just rewrite this old code.

Please. Yeah, it will do exactly the same thing, it'll just work better, no one will notice.

So I think the other thing there is, you know, we talked primarily about engineering. I think it goes both ways though. The go-to-market functions as well have to change in that world. I think the critical thing there is road mapping. And, you know, as you switch from that innovation stage where you don't quite know what you're going to do necessarily from quarter to quarter to a mode where you do understand the go-to-market, the ability to have roadmaps that are consistent solves so many problems. They're kind of the fundamental unit you need to keep engineering and sales in sync, but they also help with, you know, communicating vision and what to expect to your customers, particularly in the enterprise. I think is where this matters.

How did you do that at Looker? I mean, we didn't have a roadmap and then, you know, we needed to create a roadmap. That's not, you know, Rome wasn't built in a day, a roadmap wasn't written in a day either.

You know, I give, I will, I wanna, before I explain this, give all credit to the team who actually implemented these changes. So Sandra Orozco out there might be listening to this and she's one of the most phenomenal TPMs I've ever worked with. But at Looker, yeah, we, when I walked in, We didn't have a roadmap. We didn't have what I believe is fundamentally necessary to build a good roadmap, which is tools like Jira that engineering managers and PMs use on a day-to-day or week-to-week basis to track the work that is going on. Like, that is to say, a roadmap comes in two parts. One is like vision. It's easy to build a visionary roadmap. Roadmap, right? Like, you ever hear like Elon Musk talk about like, yeah, landing people on the moon?

The 4 sentences for Tesla, or, you know, it's like, okay.

Yeah, exactly. Someday we'll get the full self-driving, whatever. But it's easy to talk about the long, long-term roadmap. What is more challenging though is like, hey, quarter out, like, tell me what you're really going to land and can you do it consistently at a 90% confidence rate? And at— in order to do that, you have to have a certain culture of engineering and product, and you need tools to, to support that culture. So the tools that we implemented at Looker, again with the help of an entire team, were to bring in Jira as our work management tool, to bring in an engineering culture that was heavily weighted toward predictability and quality. I told the team continually, like, I will give you all the freedom and control in the world as long as you can do it predictably, right? So that sort of culture came into play. And then delivering consistently and over-communicating our roadmaps to our stakeholder teams so that they could be brought in and not be surprised about what was coming down the pipeline. We, in order to do this, I sort of put the team through a bit of a torture chamber. So I started in October and we rolled out a small version of this process, Jira. I gave them a whole strategy for how to build a whole process for how to build product strategies called DIBS. I stole that from Spotify if you want to look it up, and we ran through a 6-month product planning cycle in 2 weeks in October. And I told them, hey guys, good job on the practice round. We're going to do this one more time. So we did it back to back. We did 2 6-month planning processes back to back in order to kick off the year. And I got a lot of complaints and pushback for that. Why are we doing all this unnecessary paperwork and extra bureaucracy? But having done that twice, we landed in the new year with the team just able to execute understanding the culture we were trying to head toward, and then I was able to kind of pull back. There's nothing better than being in a leadership position where you've just trained everyone else to do all the work. So I was able to start like pulling back after we got that sort of discipline tooling and so forth in place.

Nick, we got just on that point, we got a question from the audience. Do you tie new features and tech debt work to annual or quarterly OKRs? Do you tie anything in the roadmap to OKRs?

Yeah. Yeah, absolutely. I don't know how much we're going to get into, to OKRs. I have a lot of opinions on that, but yeah, absolutely. I think philosophically, I like to have an 85% accurate prediction of what my organization is going to build within a quarter. And in order to do that, you know, you can use OKRs, you can, you can use Many constructs, like I think OKRs is, depending on where you work, one of the more popular ones. But for me, every quarter, and that doesn't mean stuff things into a quarter. What I ask teams to produce is a slightly ambitious plan, right? Given the resources that you have, tell me how you would want to deploy them. Give me a slightly ambitious plan that you feel you can hit with 85% confidence in 3 months. If something you're proposing takes longer than 3 months, we should have a discussion about how to break it down into something that's a little bit more predictable. And then ideally, you'd— what, just one anti-pattern I see is, uh, people say, oh yeah, 3 months, that sounds great, but then every single project that they propose takes exactly 3 months. So this is also an anti-pattern, just, just for EMs out in the audience who might be listening to this. If, if your estimates always land on like a month boundary or a quarterly boundary, like an experienced director will see that and assume that you've, you've kind of whiffed it. So, you know, do enough estimation to, to give a high confidence estimate, uh, within 85— with, with 85% accuracy, and that will most likely not land on exactly a quarterly boundary.

That's awesome. Okay, so let's keep, let's keep going with the, with the theme. So, you know, we talked about that transition from product-market fit, innovation to execution. You've now gotten the engineering team into a place where They are bought in and excited about involving sales in the product roadmap. You've got a product roadmap. Now you've got to scale out the team. And one of the things that I really liked about your writing is you have strong views on the difference between a manager and a leader. Can you go a bit more into detail about the differences between a manager and a leader?

Yeah, I think, look, it's not to say— by the way, whenever I answer this question, I think people interpret it the wrong way. Look, managers can be leaders and ideally, uh, ideally you get managers who are leaders, but, uh, I also want to contrast that by saying the expectations for leadership are not quite the same as what you would expect for management, like with, with managers. And we were just talking about it. What I really want is someone who can take care of their team, take care of the people on the team. All right. One, that's my high-order bit for managers. And then two, they need to be able to execute against a strategy or a roadmap with predictability and quality. Right. So management is all about building things and getting things done in ideally a sort of stable, predictable, low-risk, you know, fashion. Right. Like, that's what a manager does. Now, think about your favorite leader and and what they do, right? I pick anyone, and if they're stable, you know, they're probably not doing what most leaders are brought in to do. Leaders are often brought in to facilitate change, like to set new directions, think about the long-term horizon. They use empathy and charm and the heart to inspire people to take on new challenges. So they're really, if anything, you know, we're— we talked about managers being forces of stability and predictability. You can think about leaders in the opposite fashion. You want leaders who are going to come in and inspire people to do something different. And they don't necessarily have to— leaders, in order to get what they want done, they don't necessarily have to have people reporting into them. You think about like managers might have support, subordinates or direct reports, but anyone can be a leader as long as they can get followers, people who believe in their ideas and the vision that they're trying to project. And I think that's the fundamental difference as I would describe it. I think leaders can come from anywhere. And I try and in any organization I build, construct systems that we can go find them. And the reason I believe that's so important is because as organizations scale, the number of people obviously scales linearly. The number of managers does not scale linearly. It scales logarithmically. So if you're not tapping into every single potential leader in your organization in a systemic way, you're missing out on opportunities to find talent, change agents, etc. OK, I'll start ranting about this.

No, no, no, this is awesome. This is awesome.

So how do you—

so So, you know, as you talk about that phase, one of the key things we focus on at a board level when working with startups is the middle management becomes super important, right? As you start this transition of actually growing, and those managers are critical because you need help recruiting, you need help managing. Like you said, the number of employees in the company is going to grow by 3x, 4x, whatever it is in a given year. And so you can't just have a level of VPs and then a bunch of individual contributors and expect success, right? Yeah. You need a bunch of— and within the engineering teams, there's lots of different ways to construct it. You could bring experienced people in-house, you could promote from within. One of the things that you talk a lot about is finding out leadership opportunities within a business. How do you think about building out that? And you did it at Reddit within 18 months, which is just incredible. How do you do it? How did you grow that middle management layer to be successful?

Yeah, I mean, for folks who are listening in this, the story about Reddit was about about scaling from a startup scale. We had about like 35 people, and then we— I think we ended, we had like 170 or something like that. And we did it really, really fast. So I see this pattern happen at other startups who are experiencing hyper, hyper scale. And the way I like to think about this is two parts. First, you know, there's a whole discussion we could go into about organizational design. I won't go down that long rat hole, but you do have to think about how you want your organization to be structured. The way I like to structure teams at scale is build on a strong foundation of shared services and shared capabilities. Think about your product silos and who's going to lead those product silos. And then on top of all of this, think about teams that help you better understand your customer, like data science or growth teams. So that, that roughly is the shape that I'm aiming for. You might have a different shape. The key thing is once you have that shape, you're going to recognize that you need to bring in a certain number of managers to to fill key roles? And you should be able to identify that really quickly. And then you gotta look at your army. Who do you got working for you? You gotta ask yourself, do I have the people on staff that I'm gonna invest in to take on management roles? Or should some of these people stay in the IC track? Do I need to go external? And at Reddit, we had that problem really quickly, 'cause the day I walked in, Steve, the CEO, was like, Nick, we gotta scale up. Really, really fast. So within the first couple of weeks, I was starting to think about how do we suss out the difference between future managers and people who want to go down that IC track. And I came up with this sort of test. I call it like the— oh, you got a slide for this?

I have a slide. I forgot about the slides. All right. But here's a slide in here. Oh, that's way too big. Okay, let's see.

Oh, it's all good. I'll talk through it. It's, it's sort of a test. If you've ever seen the movie Blade Runner, they've got this thing called the Voigt-Kampff test. And The idea is you're sitting from across from someone who looks like a human, but you're not quite sure. You got to figure out if they're a human or a robot. And the Voight-Kampff test is a series of questions that the Blade Runner asks the prospective candidate. If they pass, great. If not, I guess the Blade Runner shoots them. And you can take that same sort of approach to sussing out potential managers into people who will become engineering managers or people who should maybe go down an IC or architect track. And we did this at Reddit. By the way, the way that this was structured was not a formal test per se. This was something that, you know, came up in the course of one-on-ones with people on the team. All right. So we didn't—

So you weren't walking people into a confessional and asking, you know, behind the closed curtain.

Yeah, exactly. But I mean, I think it's important at early stage startups to start putting this framing in place that you can have a management track and a purely technical track. When you, when you get later on into larger stage companies, I see companies that haven't quite solved this and they're still struggling with it. And the way that manifests is you might have, you know, a lot of people with the tech lead title, you know, I've consulted with a lot of different companies. I've worked at a lot of different places. Every single one of them had a different definition for tech lead. And for me, when I'm trying to build out an organization, I like to, to clarify that, like, you can have tech leads at certain scales of your organization, but as you climb through the ranks, I like that to grow into a dedicated IC track or dedicated management track. So these are some of the questions roughly that we asked, you know, do you care about more about people or technology? For me, again, If you're going to be a manager, people has to be the high-order bit. Thoughts about shipping toward a deadline. When I talk to— in general, when I talk to architects, they're not as focused on deadlines. They're thinking about the tech longer-term implications of the technology choices. That's what gets them excited. PM goes to two of your direct reports and asks them to immediately start working on something. If you're an engineering manager and you don't immediately shut that down, I would have some concerns. But you do have— there is a certain a certain type of engineer who will be like, oh yeah, let's just jump right into that and go figure out, start problem solving, right? And the other thing, you know, if you're gonna be in a management role, particularly, you know, in a hyperscale startup, you end up spending a lot of your time staffing and you gotta build up a thick skin by getting LinkedIn rejections. So I threw, for Reddit's purposes, I threw that in. I think, you know, as you think about this for your own companies, you're gonna come up with a different set of questions. The big takeaway from this, though, is do put in place as early as you can a way to think about these two different tracks, because if you don't, you will eventually be asked about it. You're gonna have to solve this problem, and I think it can be solved fairly early in the stage of— in the start stage of startup life.

Awesome. Touching on that last point, one of the stories that I really appreciated in your writings was talking about the interview you had at Microsoft when you were recruiting a machine learning engineer and you met a machine learning engineer who was very quiet, very technically sophisticated but very quiet, and then you were part of the group that votes on who's admitted to hiring group and you were the only no vote. Can you walk us through that story and what the lesson was there?

Yeah, so this was a— I guess I'll share all my mistakes. We'll just dig into my trove of bad decisions. Thank you, Thomas. Yeah, no, so So I think over time, my opinion on what it takes to be a leader has, has changed. And I remember this incident well because it was a moment where I— my manager came in and corrected me in my way of thinking. So I had a candidate come in who was, you know, interviewing for a machine learning technical role. And he passed all of the, of the interview questions. Well, all the technical questions just knocked him out of the park. I remember vividly because I used to pride myself— this is the old Microsoft days— on having impenetrable questions. I had dynamic programming. I would never do this today, but like back then, I guess it was more common. And this dude just came in and knocked them all out of the park. And the challenge is he's just like really quiet. His name was Tony, by the way. So Tony was really just quiet. Like, he went through the questions, but there was no charisma. And I— when we got to the interview panel, you know, went around the room and everyone gave higher and higher, higher, higher. And it got to me and I was like, no, I don't see Tony as a leader because I can't imagine how someone this quiet and reserved would be able to come in and lead. How is someone this introverted going to lead? And so I said no. And then my manager pulled me aside. And I appreciate him now for having done this. I didn't understand it in the moment. But now I definitely appreciate it. He said, Nick, I'm very disappointed in you.

Really? That's how he started?

Well, this is, this is millennial speak for you're an idiot. I appreciate him phrasing it in that way, as opposed to what I'm sure was going through his head, which Nick, stop being so dumb here. So he's like, Nick, like, really disappointed in you. Why don't we— why don't, why don't you just trust me? We'll give this guy a hire and we'll see how things turn out. And Tony ended up being one of the best hires I ever made. I worked with that guy for like another 8 years. When, when we— he brought him into the team, he started mentoring people. I mean, he, I mean, he was stunning. Like, I was completely wrong about him. And he had a way of leading that, yes, was quiet and reserved. Was also incredibly impactful. So it just taught me that, like, leadership can come in many, many styles. And that was the first moment where it really hit that you, you can't— there is no, there's no defined way to lead. I push people on this all the time. If you, if you look behind me, you'll see I've got a bunch of leadership books on my shelf. And what I tell people is You can read all of these things, but there is no formula. Like, leadership comes in so many shapes and styles. It's all dependent on the situation you're in, bringing your own personal strengths to the table and understanding, like, how you can be the person to step up in the right moment. And so, so Tony was the right person. He had, um, he had the knowledge, skills, and abilities in machine learning that allowed our team to, to thrive and get to the next level. And it's just not something that came across during the interview. So I'm much more open-minded about those things now.

Is that something that you think you now can tell within an interview, even if somebody is quiet, they've got the right leadership style? Or like, how do you sort of suss that out when you're meeting new folks?

Yeah, I think what I've since come to believe is, you know, you want to look at how far people have come and what they've overcome. You know, is a good indicator. And that may be different for different classes of folks. I won't get into that at this exact moment. But I think that I increasingly look at a combination of track record and whether or not that person has unique passion, skills, or abilities that would help them lead in the moment. And it's hard to describe. I think like whenever I do interview panels, I like to make sure I get a lot of different viewpoints on, on the candidate. But spread over a number of people, you can start to suss out whether the candidate is particularly excited or insightful or inspired by the role that you, you, you have. I don't know if that necessarily, I would say, is a requirement. But in terms of spotting potential leaders, one of the things I look for is, is that element of passion, that spark. And then when I'm trying to generate new leaders, I try and think of ways to give people that spark. Right. So not everyone has that at every moment in their career. Right. Not everyone at all times is working on the exact one thing that is going to spark their passion. But I think as a, as a leader, you have to continually be figuring out ways to give people that spark. So I'll stop there.

Yeah, you call it the blue flame, right? I know the blue flame, right? What's, what's the blue flame?

That, that is— man, you've done a lot of research. So yeah, so the blue flame, I think, is it's when you get the intersection between what someone's personal passion and motivation are, like the thing that they, they just love working on. When you get that intersected with what the business needs at a given moment, you end up in the blue flame, the hottest part of the flame. And as a, as a leader, I'm always trying to think through, you know, in one-on-one conversations or career coaching conversations with people who work for me, you know, I have this in the back of my mind. Is this person in the blue flame or are they, or could they be, should they be encouraged to, to do something else that would get get them there, right? So you can figure that out by— it's not, again, like one-on-ones are sort of the magical element for, you know, for anything you might do as a manager. You can, you can generally suss out the best of it in a one-on-one, just understanding what really inspires people. But, you know, the way I like to suss this out is to talk to people about their longer-term goals and where they see themselves. And I always keep that in the back of my mind when we're talking about product roadmapping and so forth and so on. And if I see an opportunity that someone's long-term ambitions and where they want to go intersect with what the business needs to take a bet on, then I go and try and give people that chance and see where they end up. And so far, it's been— it's worked out really great. I mean, like, if you get people on something they're passionate about, you get a little bit of extra horsepower. And then also, I think it builds a better connection between me and them as a manager. Like people want to know that you're thinking about them and have their best interests at heart as you build the organization. That's the easiest thing to lose when you're getting to big scale. I'm terrified of being in a world where I manage hundreds and hundreds of people but lose the ability to really understand and think about what makes people passionate. I feel if I lost that, I would lose myself as a leader.

Yeah, that's wonderful. Awesome. One more question from me, and then we've got loads of questions from the audience. Just to kind of wrap things up, you do a lot with an organization called DevColor, which is, you know, the mission is around diversity and inclusion engineering. It'd be great if you'd spend just a couple of minutes talking about the organization and why you're so excited about it.

Yeah, yeah, so just, guys, this, if you don't know about this organization, like, I'm gonna talk about it, but go to the website right now. So I'll talk about how I got introduced to this organization. I had been in Seattle for a very long time in my career. I spent about 15 years in Seattle, and not, not to knock folks who are from there, but it's not a particularly diverse part of the world. And I, as a Black software engineer, you know, I spent most of my career— well, all— sorry, all of my career at Microsoft without seeing another Black VP of engineering or someone in a high-level role. So I was You know, I was fairly frustrated by this. And when I moved to San Francisco, one of the first things I did was I attended a meeting of DevColor. 300 other Black engineers in the same room. It was absolutely mind-blowing for me. Like, first time in my career I had ever, ever experienced this. And from all career stages too. Other Black execs, people mid-career trying to rise up, and then people fresh out of boot camps. All of the above in one place. So I joined the organization. I got so passionate about it that they later on invited me to be on the board. With my first nonprofit board, I discovered that you have to pay them to be on the board.

For the privilege.

Yeah, well, it is what it is. Anyway. So I'm now on the board, and I encourage you to look into this organization. It is an organization that uses a squad model or lean-in circle model to help teams of Black software engineers achieve extremely ambitious goals. So when I was doing my DevCaller squad, I set my goals— I'm not making this up, by the way— I set my goals as doing more public speaking, I wanted to do more writing, and I wanted to get more connected with the venture capital community. Today, a few years later, I have done speaking events literally around the world. My blog posts and writing, which, which Tomasz has referred to here, have gotten hundreds of thousands of views. I'm an angel investor in more than 30 companies, and I've achieved all of those goals, and I track that directly back to my experience at DevCon. Of color. So shout out to them. Please go look at this. If your company is interested in finding ways to support diverse engineers in your team, like, this is the best organization. Like, period, end of story. Like, go talk to them right now. Okay, I'll stop.

Awesome. Thank you so much. Okay, going through some of the questions from the audience, there's a real quick one. What does IC Track stand for?

Oh, sorry, guys. So individual contributor. Sorry, really tactical question there. So, uh, you think about, um, if you're an individual contributor, um, you know, how do you, uh, as your career, uh, goes on, how do you continue to contribute if you don't necessarily want to be in management? So a lot of people think like, oh, if I want to climb the ranks, I have to be in management. That, that's not true. Like a well-designed, uh, if your company has a well-designed, well-thought-through career model. They'll have a ladder for people who want to go into management, which is a very, very different skill set. Trust me, right? You definitely don't want your head of your management career track messing around with, with checking things in. Trust me, at large companies, I won't get into that. And then you got, you've got your IC track where you might move up and become an architect or a distinguished engineer. But in that role, you're going to stay a little bit closer to the day-to-day of the code and and technical decisions and things like that.

Awesome. A related question: How does the relationship between an architect and an engineering manager work?

So I have an opinion on it. This works differently at every company, but I think— if you think about what engineering managers or VPs of engineering or directors of engineering are doing, they're really managing the people and the resources and trying to make sure that the people are aligned to best contribute to the company goals, whereas your architects are like diving in and solving specific technical problems or setting long-term technical strategy. So the way that I like to design organizations is that, you know, every EM director or VP of engineering is paired up with the equivalent IC technical track leader. So for me as VP of engineering, I'd want to be paired up with like a distinguished engineer or high-level principal. IC to just make sure that I understand what the hell I'm talking about, and that recurses down through the organization. So I want to see directors paired up with— it could be our— it depends how you title these people at your company, but might be an IC staff-level architect or something like that, all the way down to the EM level if possible. That's sort of an idealized state. I don't think I've ever worked at any company that was able to, to do that one-to-one mapping, but it's what I try to aspire to because it makes every pair gets— if you do that, the pairs get stronger. Like, your, your architects have a better understanding of the limitations associated with the org structure and the people. Your EMs get— don't lose the sight of the technical challenges. It's, it's a good team. It's a, it's a good unit, uh, to, to bring together to make sure that the right decisions are being made.

Awesome, great answer. Okay, going back to that, we had a bunch of questions around roadmap. How do you measure and communicate predictability for feature delivery?

So, so I can tell you what not to do, and I can— so let me actually, I'll start with what to do, and then I'll tell you what not to do, because as an engineer, there's a level of tracking that I can't tolerate— you guys have to remember that I came from an engineering background, even though I've been like CPO and done all this other stuff. The easiest way to do this is if you can get project sizes down to what I was talking about earlier, something less than a quarter. If you can agree across the board that you're going to whittle down whatever you agree to do down to less than quarter-sized chunks, and you're going to track those in some consistent way, like the way I like to do this is You know, we have engineer— engineering managers and directors on a week-to-week basis. They're going to look at the project status for the things they've committed to for that quarter and say whether or not they're on or off track. And if you agree with that premise, then it becomes fairly easy to understand whether or not your predictability is accurate or not. You can— a quarter goes by, you can look at the projects that you agreed to to ship, and did you ship them or not? And if, if that number is less than 85%, you might go back and, and adjust and so on. There's more detailed ways to do this that I will leave it to you as the listener to describe if you want to use them. I personally in life am allergic to story points, which are another way to do this. If you're really like a fan of Agile methodology, you can go read up on Story Points. Another way to do this, which I saw implemented at Microsoft, you can go a level further down. So I was talking about, hey, teams should commit to projects. You can go even further down and get individuals to commit to specific OKRs and specific project line items. And I think when you— in my opinion, when you start to get to that level of granularity, breaks up team cohesion a little bit. So I don't remember— I don't recommend you using this sort of tracking methodology all the way down to the level of a specific individual. I like to keep it at the level of the team. So the team is motivated to get projects done. I think you get more positive behavior if you measure in that way.

Awesome. Great answer. Okay. Last question, because I know you're a very busy guy. No, we got to get you back to running the engineering part of Twitter. How would you recommend a very early stage startup interview a potential CTO or VP Eng?

So it depends how early stage. Does it say anything else about that in the question?

It says— so here's a question from Ben. How would you recommend a very early startup find good technical talent, potentially a CTO? How do you vet these candidates if you are a non-technical founder.

Okay, got it. So, uh, a— if you're a non-technical founder at a very, very early stage startup trying to assess the skill sets of a CTO or VP of Eng, I recommend you use your external network to bring in people on the interview panel. So I would go to the board and I would ask them for referrals. If you— if you do that in conjunction with stakeholders who are currently at the company, right? So the implication of your question is that you may not have the necessary people in your organization to evaluate whether or not the VP of Eng is going to have the strategic and people management capabilities, or whether or not the CTO is going to have the, the technical depth. If you don't have that skill set in the building, don't be shy about asking for help from your network. The catch is, though, you do need to have you do need to have stakeholders from within your company be a part of that decision, right? So I remember when I was interviewing for, for Reddit, getting the opportunity for me to meet the existing engineering managers and tech leads was, was a very valuable sort of conversation. I think they got a sense of what my style would be like when I was going to come into the organization. At the same time, I could sort of tell they didn't, they didn't really know what a VP of Eng was supposed to do. So I don't know if they— I used that as an opportunity to explain to them, you know, so many of the things we talked about here, like, you know, what does it mean to be, to build an organization and kind of get them comfortable with what I was intending to do. But in terms of that internal group of people evaluating me, I don't know if they necessarily weighed in as much. I remember, I think Steve brought in a guy named I like to do it quite a bit, like Marty Weiner, to help evaluate my skills as a VP of Eng for that particular role. So don't be shy to ask external, is all I'm saying.

That's great advice. Nick, thank you. This has been incredible. I've learned a ton about you and really grateful for sharing all of your insights throughout your incredible career. Thank you so much for the time.

I really appreciate the invite and the chance to lose my voice with you. And if, I guess, if folks want to connect with me, you can find me on Twitter, which is an awesome product and awesome place to work.

Shameless plug.

Shameless plug. And you can connect with me @NickCald on Twitter.

Awesome. Thank you so much, Nick. Thanks to everybody for the questions.