"What Do You Mean They Quit?!" Why Engineers Leave | Plato, Change.org & Notion

Please welcome Plato co-founder and CEO Kwang Hwang, Change.org CTO Elaine Zhou, and Notion head of engineering Michael Manipat.

Good morning, everyone. Thank you for being here. Hola. Bonjour à tous. We're in Europe, right? So my name is Kwang. I'm co-founder and CEO of Playdoh. And as I've been talking to many engineering leaders, the world is scary out there. It is. It is scary. We have so many, so many different stories. Of engineering leaders that are just having a really hard time to hire engineers and more importantly, to retain those engineers. It costs lots of money to actually retain the engineers. One-third of new hires usually leave after 6 months. 18 months is actually the average tenure for an engineer in Silicon Valley. So 18 months. So just by the time you hire them, they actually already left. So that's a lot of figures that are pretty scary out there. But the more I talk with engineering leaders, the more I get some really interesting advice. And today I'm very, very happy and honored to have two of the best engineering leaders I know. They have very interesting stories. And very interesting tips to share to all of you. Michael, Elaine, do you want to introduce yourself really quick?

Sure. I'm Michael. I'm the head of engineering at Notion. And for those of you who aren't familiar with it, Notion is an all-in-one tool for tasks, docs, wikis, things like that. But beyond just those use cases, Notion wants to be a platform for building software tools in general. So you can imagine most SaaS products are built out of the same kind of primitives, an editor, databases, workflows, automations, sort of build build that platform so individuals, companies, developers can all customize our software tools on Notion. So I've been there for about 2 years. We've grown from around 40 people to around 300 over those 2 years, around 10 engineers, 125. I was at Stripe for about 6 years before Notion, where I ran the machine learning practice.

Hi, my name is Elaine Zhou. I'm the CTO at change.org. I, um, hope that you have heard about Change and all have the users of Change either signing out petitions or contributing or creating petitions. Change.org is the world's largest petition platform for people to make changes they want to see. We have over 400 million users around the globe in 196 countries or regions. So my role is manage the product and engineering teams, especially engineering. We have over about 70 people. We are completely owned by a nonprofit, so we have a very different approach and also unique challenges compared to many other large big tech companies.

What is fascinating when I talk to the two of you is you have such a different company in terms of you are an NGO, you are super high growth classic and traditional startup. and the lever you have to actually retain and engage the engineers are very different. And so that was fascinating, and hopefully we're gonna get through all of this, uh, together. So the more I talk to engineering leaders, uh, the more, uh, I see retention as a journey, a journey of the engineer. Um, it's like a product. When you try to retain your users, you have to retain them from day one, from even before they join your product or your platform. It's the same thing for engineers. You have to think about how you retain them even before they join your company, which is interesting. And then when they join your company, and then even after when you actually grow them and develop them. So we're going to go through this journey together. So the journey of an engineer and what you can do at every stage of this journey to actually retain your engineers. So let's talk about even before they join your company. So the hiring process. So this is something that actually you can start to think about retaining your engineer even before they join your company, which is interesting. So even before the interview process, tell me a bit more about how's the interviewing process and how do you scan for people who are going to stay in the long run? Anyone wants to start?

I can jump in first. For us, as you can imagine, a company like change.org is extremely mission-oriented. So we put a lot of focus on highlighting our mission, the purpose, the user impact, the impact to the world in our recruiting process, from job descriptions to the first contact from our recruiter to candidates, and then throughout the entire interview process and onboarding. So we put a lot of focus then We only want to attract people who agree with our mission, obviously, and are passionate about coming to make change the world through technology. So that's our main focus.

But everyone tell— everyone tell you that they believe in the mission, right? So how do you do really to identify those who really believe in your mission?

So we have some advantage as being a petition platform. We have hundreds and thousands active petitions running on our platform in any given day. And we, many of those petitions or campaigns reach success in any given day. So we can make big, you know, campaigns and make it to New York Times, Washington Post, various, you know, media outlets, and we always highlight that. So in every one of those contacts, you know, connections with the candidates, we highlight those stories. And sometimes we also switch to smaller petitions in the local bases, the local petitions, that's what related to the candidate, easy to connect to them. So if we know a particular country or region or location the candidate is from, we will also highlight those petitions too. That's really helpful. You can see what technology can actually make the impact. By the time they talk to engineers or talk to me, we would definitely kind of put more focus on how do we utilize technology into our platform, whether or not it's AI infrastructure or just scalability to really highlight what engineering can contribute to the mission.

Michael, how do you do to actually screen those engineers that are well aligned and those who are going to stay?

Yeah, so there are 3 things we interview for specifically in the hiring process. Just like Elaine, we start with what you might call a mission alignment check. So as I mentioned earlier, Notion is very focused on being something that allows tool builders to customize their own tools. And it's something we discuss with candidates at the beginning of the process. And it's easy to check for this. People who like tool building as engineers will have done things like, say, worked on compilers or can talk about tools they've built. Notion's in a lucky place where most engineers who join Notion have personally used it as their own tool. So that's an easy way to check, oh, these people know what we're trying to do, what we're looking for. I think the second thing we check for is just a desire to build product and not just implement technology. So we have an interview question which is very ambiguous. It's how would you roughly design this product? What are all the design choices you would make? What are the underlying technical choices you would make to see how people deal with product ambiguity? And the last thing we do at the end of the process is we have a, what we call a craft and values interview. So either I or our CEO will meet with every engineer and just talk to them about Notion's values, things like our emphasis on rigor, or on moving quickly and look for examples in people's past to see, oh, do they behave or do they live up to the values that we have in mind for the company? That way there's no divergence in the future if, you know, between our values and those of the people who join. So mission alignment, product sense, and crafting values are three things we look for in the interview process.

So mission alignment is something that you share. So what's the mission, for example, at Notion?

The mission of Notion is to to make toolmaking ubiquitous.

Okay. And you shared this with them. And how do you actually try to understand if they're real aligned with, with this? Are you just having conversation? Is it a take-home assignment or project they have to do or something? Yeah.

So it's not a take-home assignment. It's, uh, it definitely is more of a feeling or a vibe. Like I mentioned earlier, when you're talking to someone about Notion's mission is to be a platform that allows toolmakers to customize or craft their own tools. People will often be able to say, oh yes, I've used Notion myself to build this tool, say my recipe book or something like that. Or they'll have some comment about how when they were at a previous job, they were working on a lot of developer-facing tools. So it's pretty easy to grok or to glean from these conversations, you know, is this someone who cares about enabling others to build their tools? Or is it just someone who is looking for another software job?

So if they didn't use Notion, it's a deal breaker?

It's not a— I wouldn't say it's a blocker, but like I mentioned earlier, I would say 90% of people who end up joining Notion engineering use Notion in their personal lives or in a previous job. We just had an engineering manager join a few days ago who had used Notion for 3 or 4 years in his past job. So most people who join are very familiar with the tool.

90% of people you hire already use Notion before or actively using Notion before? Yeah. Do you have 90% of people who did a petition before?

I would think so, but I don't— don't quote me on it. I don't know the actual count.

No idea. So most of people also who interview at change.org actually have been using the product?

Right. Most people get a lot of our emails. Probably many of you do.

Good. OK, so let's switch to the next phase. So in order to retain your best engineers, it starts before they join. To make sure, so mission alignment, making sure they have the right traits. For example, they are product engineers, they have desire to build product, and also how they align with your values. It's, you mentioned with the CEO, co-founder of Notion and yourself and/or yourself, which is super interesting. So every single hire go through the CEO or yourself for the engineering team at least, which is just So I interview every candidate at the last stage. Do you have also some similar value alignment type of interviews? Yeah, for sure.

Good.

So let's go to the next stage. So let's continue in that journey. I'm a developer. I passed the interview. I'm using Notion. I'm a big fan. I'm using Change.org. Now, what do you do to actually reinforce and making sure that they're going to stay in the long run? It starts the onboarding, but you also have the whole process for the first 1 year, and hopefully after 18 months they don't leave. So what do you do in the first 18 months?

You want to go first?

So I think, I guess, the word in my mind is growth in 3 senses. So the first is, this is a bit trite to say, but company growth, revenue, users, valuation, is the single best way to retain engineers in this market. Anything else that you do is not going to really help if the company isn't growing, if it's plateaued. So we obviously invest a lot in making sure the product is doing well. But secondary to those are two things. One is on the growth front, you want engineers to know exactly how their work impacts the company's growth. So as an example, Notion's right now working on improving our offerings for enterprise users, you know, 1,000, 2,000, 5,000-person companies. And throughout the engineering org, the work that people are doing clearly addresses or ladders up to that. So we have people working on search to improve search in large installations. We have people working on permissions to say, make a permissions model that's more granular for enterprises. So those engineering teams can see very directly how their work impacts our goal of moving upmarket, as it were. So it's very important that engineers can see, oh yes, I am helping to move revenue numbers or achieve product goals directly. I was at Google for a few years before Stripe, and my work was, you know, light years from Google's core business. And there was no way that I was actually driving revenue there. At Notion, it's very important for us to have engineers who are really driving company goals. Just the second thing on growth is, it's interesting to me in contrast my time at Stripe to my time at Notion, where, you know, 8 years ago or 10 years ago, you were just doing work at Stripe, heads down, not really focused on personal growth as much. I find engineers today are a lot more conscious and deliberate and thoughtful about their careers. So even when Notion had 20 engineers, there was a desire for there to be a career ladder and levels so people could see how they were growing. So we actually just introduced these, uh, uh, a few weeks ago. But, uh, I think giving people a sense of tangible progress, how they're growing, what the future of the company looks like, that's really important for retention in a way that it wasn't in my mind 8 years ago or so.

So you said 3 types of growth, the growth of the company, the growth of the individual, and what was the third type of growth?

So it was growth of the company in terms of revenue and users. There was how engineers directly drive that growth from their work, and then individual career growth.

Okay, interesting. So in order to retain your engineers, you have to make them realize they, they're growing to help the company grow, and they are growing themselves thanks to career development. So let's talk about career development for a minute before I ask Elaine what she can, what she's doing also at the same journey, stage of the journey. Career development is something you introduced only a few weeks ago, so it's super fresh. Tell us a bit more about this process really quick, maybe some of head of engineering or CEO here wants to do something similar.

Yeah. So even a year and a half, a year ago on engagement surveys, probably the single thing that people wanted the most was a sense of career progression or knowing what was required of them in their role, what their expectations were and so forth. So even late 2020, we knew that there was a desire in the engineering core to have some kind of formal leveling the ladder system. So about half a year ago, We began working on that in the engineering management group. I had two of our senior engineering managers begin drafting just what do each of the seven levels correspond to, what are the expectations or requirements of each of those. That process of that initial drafting took maybe 2 or 3 months, and then there were rounds of iteration with the rest of the engineering managers earlier this year. We took a group of select ICs, had them review the ladder, and then as part of our spring performance review cycle, we actually officially rolled out the ladder, had people do peer reviews. We had a calibration and we did level delivery a week, a week or two ago. So it was a pretty long process, primarily because drafting ladders and making sure they encapsulate both industry-wide norms but also capture the unique values of your company, that just takes a while to do.

4 months. It was the total process.

Yeah, that's right.

4 months. And you did it before. At Stripe, correct?

Yes, I also went through the laddering process at Stripe.

Okay. And you saw it at Google too?

When I was at Google, ladders already existed. So yeah, I was there for the rollout, but at Stripe and at Notion.

Let's talk about what you are doing at change.org to actually have the engineers really focus on the mission and how to reinforce that over time and how you develop your engineers.

Sure.

I should say that, you know, change is slightly different. It's obviously with growth, but it's not like high growth, like a notion. But I cannot stress the importance of growth, right? If you just know growth, you can't really have the end of the way. You don't have that much leverage motivating, continue to motivate your team. But we also put a lot of focus on career development, the individual growth. For us, we measure growth with the user impact at the company level and revenue and user impact. For career development, we try to be extremely creative, utilize the very limited resources we have as a nonprofit. So we bring more focus on individuals' desire, what area of development they want to, what can we use if we you've helped that individual to grow, what can that person contribute to the company, so make them match. So we have different kind of development needs. For example, we do coaching. Those more for more senior directors or senior managers for specific needs, and we identify external coach in the short-term period that have very targeted area to focus. We have mentorship, we also use Plato as well. Those are more target mid-level managers, senior ICs who want to develop leadership skills. We also putting together right now, we're actually rolling out internal engineering mentorship program to utilize, maximize our own staff's experience and expertise, and then match with more junior, younger engineers to create this 6-month or 9-month-long mentorship program that would be a very tailored and personalized growth. And then we also have a lot of training programs. User training program, we do the train the trainer, meaning we send a small number of individuals, whether it's product manager, engineers, engineer managers to really well-created online training courses or in-person trainings. And then they come back and share. And those training could spread into from security to engineering, engineering management, to all the way to product and design as well. It's incredible. So we're limited. We do all of this, and we have to, you know, We, it's a pretty challenged market for us to compete. As a nonprofit, we don't have these potential stock options, you know, in the Silicon Valley everybody expecting stock options, equities. We don't have that in our package, right? So we have to add on to other things to attract people. So, what I'm trying to say is beyond the growth, we have to really work on engagement with our staff.

It's fascinating. You are an NGO, so obviously you don't have stocks, and it's not something that compounds and gets a lot of value over time for employees. So you can't use that to retain your engineers. Obviously you can, but we all can use growth of individuals, growth of employees thanks to levels, so you can see a progression in your career, or growth thanks to coaching, external mentorship with platform like Plato, for example, or internal mentorship or even trainings, all the things that you're doing, which is pretty incredible. There's definitely a culture of high growth of individuals at Change.org, would you say?

Yeah, we try very hard to build out the company-wide growth mindset, right? I mean, there's plenty of research said about happiness come from you know, mindfulness and curiosity, right? Curiosity drive, you know, require growth mindset, and that's very important. And the other thing, the last thing people want is working for a purpose. Yeah, so that's the mission focus for us. So you got to work in all, all of these areas to deepen the engagement of your staff.

Yeah, there is a topic we hear all the time in our mentorship platform is about, if the levels, should be public or private, if we should have the open salaries or not within the company. I know you have a different point of view on this. Can you tell, uh, uh, the, the crowd what, like, what you think about, about this?

Well, so I, I can, I, I can start. So I was at Google where levels are public. There's actually a single webpage at Google. I don't know. it was 10 years ago now, but go to go/endstaff and see the level of every single person at the company. Stripe and Notion both have private levels, so the level is known to individuals but not to the wider population. It's not publicized. And I think I prefer that for two reasons. You know, one, if you make levels public, it's a trapdoor decision, so you can't walk it back if there end up being secondary consequences you don't like. But second, we want the focus in the workplace to be on the content of the work. If you're having a discussion about a tech spec or a product review document or a product requirements document, you know, let the discussion stand on the merits of the discussion and have not, like, we don't want people to fixate on, oh, this person is debating with me and she is at a higher level than I am, so perhaps I'll just be quiet or see the argument too quickly. So we're definitely worried about these types of, uh, effects if people are too aware of levels. There's obviously a counterargument where people want to be able to publicize their career growth and their progression. So then pass off to Elaine for the opposite view.

Yeah.

So I think this is a highly debated topic. We more lean toward keep transparency is important to us. So we have company-wide standardized levels, meaning every department, we actually have entire company levels and through entire, you know, collaborate throughout the entire company. So the levels and the requirements for each levels are public to all the staff and employees. Salary band is public, and we actually list our salaries in the job posting. So we actually always lean toward transparency. We do have the title that match the levels, so effectively the level is more or less public. There might be a few variations, but yeah, we definitely lean toward—

So transparency, is it helping retention or not?

I think transparency help engagement. Yeah. Because we put a lot of focus on diversity, inclusion, inclusion and equity, right? Equitable, uh, you know, require transparency and try to make sure that, uh, non-dominant groups, you know, staff from non-dominant backgrounds get equal treatment and rewards. So that is important for us. So it's a very different approach, but they're all trying to work in the similar goal though.

You agree?

I think it's, both systems can work with you, you know, on the right way. I will say, you know, at Notion, we don't ask people what their levels were at their previous employers when we interview them. So we try to assess purely on the merits. You know, when I was interviewing for the role at Notion, no one ever asked me for a resume or my level at Stripe, for example. So I don't think that levels are actually as important in terms of career portability or growth as people might think.

Um, I— we also running out of time, so, uh, let me, uh, just recap a little bit throughout this journey, uh, really quick. So in order to, uh, retain, uh, your engineer, so, uh, we need to focus on retaining your engineers, uh, from day one. So even before, uh, we hire them, so how do we do to actually, um, interview on a mission alignment, on values, on type of engineers that you want, how much product-oriented they are, etc. And the second thing is really throughout this journey how you actually empower them to actually be better and better in their roles. So we discussed a lot about coaching, about mentorship, about training, but also about empowering them on what that what they are doing is actually helping on the growth of the company. For example, you mentioned the enterprise features, and people don't want to leave if they are going to be aligned with the mission. And for you, it's really about— YouTube is about the mission, YouTube is about the growth of the individuals, and for you it's mainly about the growth of the company and the eventually the growth of their stocks. So throughout this journey, there is not one thing to do, it's multiple things at every stage of their journey. So this, those are the main takeaways. Thank you so much.

Thank you.