Michael Seibel's approach to running a product development cycle at an early stage startup

We didn't actually really learn how to do a product development cycle until we failed at it for about five or so years. And during that time, this is what bad product development cycle looks like. One, we would release every three months for a web only product. That is horrible. Second, we would have a product meeting and we wouldn't write anything down, right? It was just four of us. Can't you remember? You're an idiot if you can't remember a conversation four people had, right? And if you forget something, just ask one of the other four people in the room, right? No. So as a result, during the first month of the dev cycle, we'd all go off working on slightly different versions of the thing we wanted to build because we didn't write down the spec. Then at the end of that month, we'd come together and we'd be like, oh, wait, this isn't. We're not really building all the same thing. And then we'd have another product meeting where we didn't write anything down and go off and build again for another month. At this point, right? Two months in, we probably have about three weeks of productivity and about five weeks of just stuff that's going to have to be thrown away. At this point in, we kind of come back together and realize that we're not 2/3 of the way done through this sprint. We're less than one third of the way done and we're starting to get sick and tired of this feature that we're building. So then we basically say, all right, slash and burn. Let's just make a shitty version of it. And we take another month to do that. Now, we've worked on this product for three months. If you had any good or new or interesting ideas during that three month period of time, you were told, we're already working on something else, so your ideas are worthless. Just write them down somewhere. Whatever. We're working on this thing right now. At the end of the three months, instead of wanting to iterate, we were sick of the goddamn feature we just spent three months building poorly. So we would launch it and if it wasn't used right away, we would come up with some new brainstorm on some brand new feature that would rescue the company. This is the wrong way to run a company. It was absolutely horrible. I was talking to Jeff earlier. The major product decisions that Justin TV made that carried through to Twitch to today was chat on the right, video on the left. We decided that in 2006 it is the same way. 2018, the vast majority of the Product decisions we made were horrible and never saw the light of day because they went through a process like this. So if your process revolves around arguing, revolves around not writing spec, revolves around long dev cycles, you are doing it wrong. You are 100% doing it wrong. What I'm going to give you is a model of how we figured out how to solve the problem. Steal as much or as little of this as you want, but understand that if you have any of the symptoms I'm talking about, you need to solve them or else your company is just going to be much less productive than it could be. First, you need to actually have a number that you track that reflects how good your company is doing. Almost always, if you ever are going to charge money to your customers, this number should be revenue. Almost always. If you are never going to charge your customers money like Facebook, then maybe it should be a usage based metric like how often do your customers come back every day. Like DAUs, it is almost always one of those two usage. If you will never charge the customers money, if you will charge the customers, many people will invent reasons why these two metrics don't apply to their business. 1% of them might be right, 99% of them are probably wrong. Whatever this KPI goal is, make sure that you're measuring it. Make sure that everyone in your company knows what this goal is every day. Helpful to put it on some screen somewhere. If an investor asks you what your KPI is, you not only should be able to say what it is to be able to say what the metric is, you say where the metric is now, where it was three months ago, where it was when you started. This is kind of table stakes. The next thing that we would do is as kind of product person, I would come into the meeting and I would say, this is the KPI we're looking to improve this cycle. At socialcam, the top level KPI was daus. And the three ways that we thought we contributed to DAUS was either new users, retention of users and new content created those three things. So every cycle we ran one of those three numbers. Moving that number to the right direction was the goal. And we run an open brainstorm. Brainstorm for us would take a couple hours and it was a real brainstorm. It wasn't a brainstorm where like you say what about this? And your co founder says that's a dumb idea. That's not a brainstorm. The real brainstorm is that any idea that's stated is written on the board. The cool thing about these brainstorms is that everyone's computers were always open to Mixpanel. So if you had an idea or you had a thought, you could always just go in and check the metrics and see like, is that right or is that wrong? You'd be surprised at how much value there is in seeing your idea on the board. Not everyone's going to get to have built what they want to have built. But the fact that your idea was considered and added to the board actually makes people feel a lot better than otherwise. People feel horrible when their ideas are shot down. CEOs, their job is to make their employees not feel horrible all the time. Sometimes I think CEOs think their job is to shoot down ideas. It's not, it's not going to help you at all. And everyone, by the way, in our company participated in this brainstorm. At that point that was four people. So easy to do. The next thing was we did what's called easy, medium, hard. So our brainstorm was actually typically split up into three categories. New features or iterations on existing ones, bug fixes and or other maintenance and tests, AB tests we want to run. We have a whole board filled out with ideas on these three categories and then we go through and do what's called easy, medium, hard. For us, hard meant it would take one engineer most of the dev cycle to build. Medium typically meant to take a day, two days. Easy means we could do multiple in a day. This is extremely important. How many of you in this room do not know how to write code? Raise your hand. There we go. I am one of you. It's extremely hard if you don't know how to write code to figure out whether your idea is easy to build or hard to build. That's something that you actually learn as a skill over time. And this process basically is the process that can help educate you. It turns out that easy ideas get built way faster, way more quickly than hard ideas. And it turns out that most hard ideas can be restated as an easy idea if you just understand what bits of your hard idea are both useless and hard. And most of the time there are useless and hard bits and hard ideas that can just be removed. And so for us this was educating everyone on the team. And for us we had a cross functional team. So someone might not realize that this is really hard on the video system. It might be an easy web future, but hard on the video system or vice versa. So this was basically educating everyone on the team on what's easy, medium, hard. It also created an objective standard by which to start thinking about these ideas instead of just based on the argumentability of the person delivering them. It was like, well, your idea is like really fricking hard and seems like it wouldn't move the numbers that much based on Mixpanel. Whereas like this other person's idea is super easy and probably can move numbers a lot. The next thing we would do is we decide hard first. So we look at all the hards and we say which hard is going to impact the KPI the most? And then we move to easys, and then we move to. I'm sorry, then we move to mediums and then we move to easys. What was interesting is that like just with the ideas on the board and with Easy, Medium, hard, a lot of the ego was removed from the debate because one, you knew your idea had been considered and two, you'd send some objective measure about how hard it was. And three, because the board has a bunch of ideas on it now, it's probably pretty easy for you to find an easy idea that you really like. And so you're just gonna be excited that that's probably gonna get in and your really hard idea, that's fine. If it doesn't. The next step is you have to write this back. This is where everyone fucks up. The meeting might be going on for four hours now and this is the step no one likes. You actually go through and you actually write down. What do we mean by we're adding video filters to socialcam? What do we mean by we're allowing people in Justin TV and Twitch to chat with one another? What does that actually mean? How is it gonna work? This is really important. And once this is done, you can then distribute tasks to the team. Now we would run these cycles every two weeks at socialcam because back then submitting to the app store took longer. If you're doing a pure web product, you can run these cycles once a week. The rule that we had to make the team not hate these really long meetings. This was the only meeting we had. This is the only meeting. And so it might take two hours, it might take six. But for that two week period of time, there were no other meetings. In fact, for me, being a non coder, my number one job was to shut the fuck up because I could create a problem, right? We're busy working, we have this written spec, everyone knows what they're doing. If I have some brilliant idea during that two weeks, right, that'll throw the whole thing into chaos. Suddenly the rinse spec's not important back to the drawing boards, or we're changing things, yada, yada, yada. What I had to realize is that every two weeks we do this over again. So for that burning idea, just fucking wait two weeks and we're gonna have the meeting again, and then we can get it in. And it turns out your burning idea is probably wrong. So it's totally fine to wait two weeks to try to convince people do something wrong. It's totally fine. As opposed to like, having this cadence meant every two weeks we had success every two weeks. If we built what we said we were going to build, we felt good. And then that cadence meant that we go into the next cycle and do even more. This cadence is extremely important because it's going to take you guys a long time to find product market fit. You'll be trying a lot of things, iterating a lot, and if that process doesn't feel fun, you're going to get very frustrated. This made the process feel fun because we had goals and we accomplished them.