"What Do You Mean They Quit?!" Why Engineers Leave | Plato, Change.org & Notion

Please welcome plato co founder and CEO.

Quang huang, change.org cto elaine zhou and.

Notion head of engineering michael manapat.

Good morning everyone. Thank you for being here. Hola. We're in Europe, right? So my name is Kwong. I'm co founder and CEO of plato. And as I've been talking to many engineering leaders, the world is scary out there. It is, it is scary. We have so many, so many different stories of engineering leaders that are just having a really hard time to hire engineers and more importantly, to retain those engineers. It costs lots of money to actually retain the engineers. One third of new hires usually leave after six months. 18 months is actually the average tenure for an engineer in Silicon Valley. So 18 months. So just by the time you hire them, they actually already left. So that's a lot of figures that are pretty scary out there. But the more I talk with engineering leaders, the more I get some really interesting advice. And today I'm very, very happy and honored to have two of the best engineering leaders I know. They have very interesting stories and very interesting tips to share to all of you. Michael, Elaine, do you want to introduce yourself really quick? Sure.

I'm Michael. I'm the head of engineering at Notion. And for those of you who aren't familiar with it, Notion is an all in one tool for tasks, docs, wikis, things like that. But beyond just those use cases, Notion wants to be a platform for building software tools in general. So you can imagine most SaaS products are built out of the same kind of primitives. An editor, databases, workflows, automations. So we're trying to build that platform so individuals, companies, developers can all customize our software tools on Notion. So I've been there for about two years. We've grown from around 40 people to around 300 over the two years. Around 10 engineers, 125. I was at Stripe for about six years before Notion where I ran the machine learning practice.

Hi, my name is Elaine Zhou. I'm the cto@change.org I hope that you have heard about Change and have the users of Change either signing our petitions or contributing or creating petitions. Change.org is the world's largest petition platform for people to make changes they want to see. We have over 400 million users around the globe in 196 countries or regions. So my role is manage the product and engineering team. Special engineering. We have over about 70 people. We are completely owned by a nonprofit. So we have a very different approach and, and also its unique challenges compared to many other large big Tech companies.

What is fascinating when I talk to the two of you is you have such a different company in terms of you are an ngo, you are super high growth, classic and traditional startup, and the lever you have to actually retain and engage the engineers are very different. And so that was fascinating and hopefully we're going to get to all of this together. So the more I talk to engineering leaders, the more I see retention as a journey, a journey of the engineer. It's like a product. When you try to retain your users, you have to retain them from day one, from even before they join your product or your platform. It's the same thing for engineers. You have to think about how you retain them even before they join your company, which is interesting. And then when they join your company and then even after, when you actually grow them and develop them. So we're going to go through this journey together. So the journey of an engineer and what you can do at every stage of this journey to actually retain your engineers. So let's talk about even before they join your company. So the hiring process. So this is something that actually you can start to think about retaining your engineer even before they join your company, which is interesting. So even before the interview process. Tell me a bit more about how the interviewing process and how do you scan for people who are going to stay in the long run.

Anyone wants to start, I can jump in first for us, as you can imagine, a company like Change.org is extremely mission oriented. So we put a lot of focus on highlighting our mission, the purpose, the user impact, the impact to the world in our recruiting process, from job descriptions to the first contact, from our recruiter to candidates, and then throughout the entire interview process and onboarding. So we put a lot of focus then. We only want to attract people who agree with our mission, obviously and a passion of outcome to make change and the world through technology. So that's our main focus.

But everyone tell you that they believe in the mission, right? So how do you do really to identify those who really believe in your mission?

So we have some advantage as being a petition platform. We have hundreds and thousands active petitions running on our platform in any given day. And we many of those petitions or campaigns reach success in any given day. So we can make big campaigns and bake it to New York Times, Washington Post, various media outlets. And we always highlight that. So in every one of those contacts, connections with the candidates, we highlight those stories and sometimes we also switch to smaller petitions in the local basis because the local petitions, you know, that's what related to the candidate, easy to connect to them. So if we know a particular country or region, location the candidates from, we will also highlight those petitions. So that's really helpful as you can see what technology can actually make the impact. By the time they talk to engineers or talk to me, we would definitely kind of put a more focus on how do we utilize technology into our platform, whether or not it's AI infrastructure or just scalability, to really highlight what engineering can contribute to the mission.

Okay, Michael, how do you do to actually screen those engineers that are below and those who are going to stay? Yeah.

So there are three things we interview for specifically in the hiring process. Just like Elaine, we start with what you might call a mission alignment check. So, as I mentioned earlier, Notion is very focused on being something that allows tool builders to customize their own tools. And it's something we discuss with candidates at the beginning of the process. And it's easy to check for this. People who like tool building as engineers will have done things like, say, worked on compilers or can talk about tools they've built. Notion's in a lucky place where most engineers who join Notion have personally used it as their own tool. So that's an easy way to check. Oh, these people know what we're trying to get, what we're looking for. I think the second thing we check for is just a desire to build product and not just implement technology. So we have an interview question which is very ambiguous. It's how would you roughly design this product? What are all the design choices you would make? What are the underlying technical choices you would make to see how people deal with product ambiguity? And the last thing we do at the end of the process is we have what we call a craft and values interview to either I or our CEO will meet with every engineer and just talk to them about notions, values, things like our emphasis on rigor or on moving quickly. And look for examples in people's past to see, oh, do they behave or do they live up to the values that we have in mind for the company. That way there's no divergence in the future between our values and those of the people who join. So mission alignment, product sense, and crafting values are three things we look for in the interview process.

So mission alignment is something that you shared. So what's the mission? For example, at Notion?

The mission of Notion is to make homemaking ubiquitous.

Okay, and you shared this with them, and how do you actually try to understand if they're real aligned with this? Are you just having conversation Is it a take home assignment or project they have to do or something?

Yeah, so it's not a take home assignment. It definitely is more of a feeling or a vibe. Like I mentioned earlier, when you're talking to someone about Notion's mission is to be a platform that allows toolmakers to customize your path their own tools. People will often be able to say oh yes, I've used Notion myself to build this tool, say my recipe book or something like that. Or they'll have some comment about how when they were at a previous job they were working on a lot of developer facing tools. So it's pretty easy to grok or to glean from these conversations. Is this someone who cares about enabling others to build their tools or is it just someone who is looking for another software job?

So if they didn't use Notion, it's a deal breaker.

It's not a. I wouldn't say it's a blocker, but like I mentioned earlier, I would say 90% of people who end up joining Notion Engineering use Notion in their personal lives or in a previous job. We just had an engineering manager join a few days ago. Could use Notion for three or four years in his past job. So most people who join are very familiar with the tool.

90% of people you hire already used Notion before or actively using Notion before.

Yeah.

Do you have 90% of people who did a petition before?

I would think so, but don't quote me on it, I don't know the actual cant.

So most of people also who interview@change.org actually have been using the product. Right.

Most people get a lot of our emails. Probably many of you do.

Good. Okay, so let's switch to the next phase. So in order to retain your best engineers, it starts before they join to make sure. So mission alignment, making sure they have the right threats. For example, they are product engineers, they have desire to build product and also how they align with your values. You mentioned with the CEO, co founder of Notion and yourself and. Or yourself, which is super interesting. So every single hire go to the CEO or yourself for the engineering team at least, which is.

So I interview every candidate at the last stage.

Do you have also some similar value alignment? Yeah, sure. Yeah. Good. So let's go to the next stage. So let's continue in that journey. I'm a developer, I passed the interview, I'm using Notion, I'm a big fan, I'm using change.org now. What do you do to actually reinforce and making sure that they're going to stay in the long run, it starts with, with the onboarding, but you also have the whole process for the first one year and hopefully after 18 months you don't leave. So what do you do in the first 18 months?

You want to go first.

So I think, I guess the word of my mind is growth in three senses. So the first is this is a bit trite to say, but company growth, revenue users, valuation is the single best way to retain engineers in this market. So anything else that you do is not going to really help if the company isn't growing, if it's plateaued. So we obviously invest a lot in making sure the product is doing well. But secondary to those are two things. One is on the growth front. You want engineers to know exactly how their work impacts the company's growth. So as an example, Notion's right now working on improving our offerings for enterprise users. You know, 1,000, 2,000, 5,000 person companies and throughout the engineering. Org, the work that people are doing clearly addresses or ladders up to that. So we have people working on search to improve search in large installations. We have people working on permissions to say, make a permissions model that's more granular for enterprises. So those engineering teams can see very directly how their work impacts our goal of moving up market, as it were. So it's very important that engineers can see, oh yes, I am helping to move revenue numbers or achieve product goals directly. I was at Google for a few years before Stripe and my work was light years from Google's core business and there was no way that I was actually driving revenue there at Notion. It's very important for us to have engineers who are really driving company goals. Just the second thing on growth is it's interesting to me, in contrast my time at Stripe to my time at notion, where eight years ago or 10 years ago, you were just doing work at Stripe, heads down, not really focused on personal growth as much. I find engineers today are a lot more conscious and deliberate and thoughtful about their careers. So even When Notion had 20 engineers, there was a desire for there to be a career ladder and level so people could see how they were growing. So we actually just introduced these a few weeks ago. But I think giving people a sense of tangible progress, how they're growing, what the future of the company looks like, that's really important for retention in a way that it wasn't in my mind eight years ago or so.

So you said three types of growth. The growth of the company, the growth of the individual, and what was the third type of growth?

So it was growth of the company in terms of revenue and users. There was how engineers directly derive that growth from their work and then individual career growth.

Okay, interesting. So in order to retain your engineers, you have to make them realize they're growing to help the company grow. And they are growing themselves thanks to care development. So let's talk about care development for a minute before I ask Elaine what she's doing. Also at the same stage of the journey, career development is something you introduced only a few weeks ago. So it's super fresh. Tell us a bit more about this process really quick. Maybe some of the head of engineering or CEO here wants to do something similar. Yeah.

So even a year and a half, a year ago on engagement surveys, probably the single thing that people wanted the most was a sense of career progression or knowing what was required of them in their role, what their expectations were and so forth. So even late 2020, we knew that there was a desire in the engineering core to have some kind of formal leveling ladder system. So about half a year ago we began working on that in the engineering Management group. I had two of our senior engineering managers begin drafting. Just what do each of the seven levels correspond to? What are the expectations and requirements of each of those? That process of that initial drafting took maybe two or three months. And then there are rounds of iteration with engineering managers. Earlier this year we took a group of select ICs, had them review the ladder and then as part of our spring performance review cycle, we actually officially rolled out the ladder, had people do peer reviews, we had a calibration and we did love delivery a week, a week or two ago. So it was a pretty long process. Primarily because drafting ladders and making sure they encapsulate both or industry wide norms, but also capture the unique values of your company, that just takes a while to do.

Four months was the total process. Yeah, that's right, four months. And you did it before at Stripe, correct?

Yes, I also went through the laddering processes. Right, okay.

And you saw it at Google too.

When I was at Google, ladders already existed. So yeah, I was there for the rollout. But it's tripod ad notion.

Let's talk about what you are doing@change.org to actually have the engineers really focus on the mission and how to reinforce that over time and how you develop your engineers. Sure.

I should say that in a change is slightly different. It's obviously we grow, but it's not a high growth notion. But I cannot stress the importance of growth.

Right.

If you just know Growth you can't really have, you don't have that much leverage. Motivate and continue to motivate a team. But we also put a lot of focus on career development, the individual growth. For us, we measure growth with the user impact at the company level and revenue user impact for career development. We try to be extremely creative, utilize a very limited resource we have as a nonprofit. So we bring into more focus on individuals desire, what area of development they want to, what can we use? If we help that individual to grow, what can that person contribute to the company? So make them match. So we have different kind of development needs. For example, we do coaching those more for more senior directors or senior managers for specific needs. And we identify external coach in the short term period that have very targeted areas with focus. We have mentorship. We also use PLATO as well. And those are more target mid level managers, senior ICs who want to develop leadership skills. And we also putting together right now we're actually in the, you know, rolling out internal engineering mentorship program to utilize maximize our own staff since you know this experience and expertise and then match with more junior younger engineers to create this six month or nine month long mentorship program that would be a very tailored and personalized growth. And then we also have a lot of training programs, user training program. We do the train the trainer, meaning we send a small number of individuals, whether it's product manager, engineers or engineer managers to really well created online training courses or in person trainings. And then they come back and they share and those training could spread into from security to engineering, engineering management to all the way to product and design as well.

It's incredible.

So we're eliminated if we do all of this and we have to, you know, it's a pretty challenged market for us to compete as a nonprofit. We don't have these potential stock options. You know, in the Silicon Valley, everybody expecting stock options, equities, we don't have that in our package.

Right.

So we have to add on to other things to attract people. So what I'm trying to say is beyond the growth, we have to really work on engagement with our staff.

It's fascinating. You are an ngo, so obviously you don't have stocks and it's not something that compounds and get a lot of value over time for employees. So you can't use that to retain your engineers. Obviously you can, but we all can use growth of individuals, growth of employees thanks to levels so you can see a progression in your career or growth thanks to coaching external mentorship with Platform like Plato for example, or internal mentorship or even trainings, all the things that you're doing, which is pretty incredible. There's definitely a culture of high growth of individuals@change.org yeah.

We try very hard to build out the company company wide, the growth mindset.

Right.

I mean there's plenty of research said about happiness come from, you know, a mindfulness and curiosity.

Right.

Curiosity drive, you know, require growth mindset and that's very important. And the other thing, the last thing people want is working for purpose.

Yeah.

So that's the mission focus for us. You got to work in all of these areas to deepen the engagement of your staff.

Yeah. There is a topic we hear all the time in our mentorship platform is about if the levels should be public or private, if we should have the open salaries or not within the company. I know you have a different point of view on this. Can you tell the crowd like what.

You think about this so I can start? So I was at Google where levels are public. There's actually a single webpage at Google 10 years ago now. But going to go nstaff and seeing the level of every single person at the company. Stripe and notion both have private levels. So the level is known to individuals but not to the wider population publicized. And I think I prefer that for two reasons. You know, one, if you make levels public, it's a trapdoor decision so you can't walk it back if there end up being secondary consequences you don't like. But second, we want the focus in the workplace to be on the content of the work. If you're having a discussion about a tech spec or a product review document or a product requirements document, let the discussion depend on the merits of the discussion. And how about we don't want people to fixate on oh, this person is debating with me and she is at a higher level than I am. So perhaps I want to speak quiet or see the argument too quickly. So we're definitely worried about these types of effects. If people are too aware of levels, there's obviously a counter argument where people want to be able to publicize their career growth and their progression. So then pass off to Elaine for the opposite view.

Yeah.

So I think this is a highly debated topic. We more lean toward keeping transparency is important to us. So we have company wide standardized levels, meaning every department. We actually have entire company levels and through entire, you know, collaborate throughout the entire company. So the levels and the requirements for each levels are public to all the staff and employees. Salary ban is public and we actually list our salaries in that job posting. So we actually always lean toward transparency. We do have the title that match the levels so effectively the level is more or less public. There might be a few variations but yeah, we definitely need to.

So transparency, is it helping retention or not?

I think transparency help engagement because we put a lot of focus on diversity, inclusion and equity, equitable require transparency and try to make sure that non dominant groups, staff from non dominant backgrounds get equal treatment and rewards. So that is important for us. So it's a very different approach but they're all trying to work in a.

Similar goal though you agree?

I think it's both systems can work with you on the right way. I will say at notion we don't ask people what their levels were at their previous employers when we interview them. So we try to test purely on the merits. When I was interviewing for the role of notion, no one ever asked me for a resume or my level at Stripe, for example. So I don't think that levels are actually as important in terms of career portability or growth as people might think.

We also running out of time. So let me just recap a little bit throughout this journey really quick. So in order to retain your engineer, so we need to focus on retaining AO engineers from day one, so even before we hire them. So how do we do to actually interview on mission alignment, on values, on type of engineers that you want, how much product oriented they are, etc. And the second thing is really throughout this journey how you actually empower them to actually be better and better in the halls. So we discussed a lot about code coaching, about mentorship, about training, but also about empowering them on what, what they are doing is actually helping on the growth of the company. For example, you mentioned the enterprise features and people don't want to leave if they are going to be aligned with the mission. And for you it's really about YouTube. It's about the mission. YouTube is about the growth of the individuals and for for you it's mainly about the growth of the company and eventually the growth of their stocks. So throughout this journey there is not one thing to do. It's multiple things at every stage of their journey. So those are the main takeaways. Thank you so much. Thank you. Sam.