Measure everything | Yash Nelapati

So anyway, another most important lesson, perhaps the last most important lesson I'm going to share today, is measure everything. Everything, anything that you can measure as a business should be measured today. Early on, we just didn't have any measurements. We went to investors and like, hey, we need some money. And like, hey, how's your platform growing? How's your product doing? How many minutes they're on your site? What do people do on Pinterest? And we're like, we don't know. We just don't know. So today we measure everything. Every time they leave? Yeah, so what we do is we buffer all the events into one big chunk, and then we compress them, and then ship it every 90 seconds, 60 seconds, and stuff. I mean, that's today. When you're building a brand new company, you can't do that. That's going to be like— you don't need scrolling data and all, right? If you're big enough like Google or something like Pinterest, decent size, you want to know why is the user clicking the 3rd link and not the 1st one. That would be the input to the search team, like, hey, I think the algorithm is bad or something. We need to get this link to the top. So data like that is super valuable at this side. Initially, I would say measure every interaction. Like, hey, they clicked on this band. They searched. And you built your own technology to label that? Yeah. Yeah. Yep. Pretty much we wrote an endpoint. We shipped this huge JSON blob. Every line is an interaction, like scroll, pinch, stop, click, close up, exit, close the app, background. So we shipped that. And then we have the data team doing all the magic. I have no idea what they do, but like, they just like compress all this, put it into S3, and then we have another team picking it up. They like generate huge reports. They write in LaTeX. I don't even know what LaTeX is, but like, it's a beautiful format. Yeah. Is this a JSON block of actions?

Yep, yep.

But you chunk it up in like, action, action, action, and then you send it? Yes. It used to be JSON, now it's Thrift.

But yeah.

So that's how we measure today. But if I were to redo Pinterest again and answer these investors, I would kind of use a service like Segment or something. They just provide a beautiful SDK API. Just scroll, click, exit, open the app. Just store your data there. I would say just raise more money for it, but it's super valuable data. You should always build measurement from day one, but it took us 2 years to realize that. Then you can build these beautiful graphs for your investors. Like, hey, this quarter we grew by 18%, or like 12%, or 0%. That's how, I mean, yeah. It's just an example, because you want to give graphs like this that would definitely make an investor make a judgment. Like, yep, I'm going to bet on this because it's I think the curve is going up and I definitely want my money in there. So if the question is like, what do we measure? The answer is always everything. I mean, depends on the size of your company and like the amount you're trying to put on your data. So what do we get by measuring out of everything? Beautiful dashboards like these. Like what country, like By what gender, by what kind of user. Is he a core user or he's like a dormant user? Is he a power pinner or like a power searcher? And like, I don't know what that— but yeah, just like slice and dice your data on different attributes on the fly. So these are the kind of things you want to see on your company and like make some judgments. Like, hey, I think this quadrant is growing too fast. We need to do some special behavior for them. And this is the feedback I hear. Should we do this? Again, go back to the data. How many people actually care about this? Oh, only the country Taiwan or something cares. That's a small portion, so maybe we could put it off into the back burner for someday we'll build that. And how every experiment is doing. How are the core daily active users, mobile daily active users? How many people are going to the close-ups? Oh yeah, people— People click a shit ton. That's good. Oh, people don't actually comment. Like, oh, maybe that's bad. Like, maybe you should kill that feature. Just make the app fast and like smaller, simpler. So yeah, it's good to build dashboards like this.

Yeah?

Is this more for like Google or like—

because I come from Europe and there's kind of social—

Yeah. I don't want to really answer that because you know the answer. But yeah, I mean, like the way I see it is like a lot. No offense to guys, but like guys don't really like plan, think about it. They just go to the store. Like, I'm going to buy this shirt. So yeah, they're not obsessed about, I want to buy my best shirt. It's just like they go, they pick one thing, they're done. But more— the product definitely gels more into feminine group because they think, obsess, and want to make additions. Yeah, thank you.

So yeah.

Following that question, I'm curious.

Yes. Because most of the contents in the front page are those with beautiful pictures and beautiful layout. So I'm wondering if that is— you guide your customers to like hang those contents with this kind of visual outlet, or is that people just like it in this current website?

Yeah, that is definitely the experience we want you to have, which is why we put the best content for you out in the open. Is that what you're trying to ask, or sorry?

Yeah, I'm just wondering if it's like the customer themselves prefer the content with beautiful layout.

So when you say customer, are you meaning like a pinner who's visiting the website? So what's the question here? Can you repeat the customer?

So I'm wondering if it's you guide the customers, guide the painters to paint those things with more visual layout, or it is the customer, the painter themselves prefer those kind of contents?

It's all, I think I would say it's like 50/50 because we kind of, we're like, hey, I think your intent is about cooking, so let me just like, guide you through the cooking experience. So we show cooking pins, and then we kind of want you to pin it and then become part of the ecosystem. So we kind of make a guess on your intent, we show content, and then we measure, like, huh, the user just closed the tab, so we got the intent wrong. How can we make it better? So yeah, I mean, there's a lot of stuff we do behind the scenes to get your intent. and all. Because websites have these UTM campaign tags that we could read and make a decision about. Or like Pin It button, like, oh, the user tried to click that button and came here. And we think this pin is about these 6 things. We don't know which things. So we're going to kind of do a good mix and match and decision making. Like, hey, I think it could be about pinning or food photography or— something else, like utensils or something. So kind of do our best job and we kind of measure it. So it's always like somewhere in the middle. We want you to pin and contribute it back to the ecosystem and make the cooking interest or topic stronger with your input, right? So this is another dashboard, the same thing, but measuring on core users. How are your power users pinning or creating content? So as you can see, comments is still low, so maybe you should kill comments someday. So if I were an engineer, this is what I would read my dashboards and make additions. Yeah, building dashboards is really important, worth it. So finally, I just want to tell, building a company is It's a loop. Like, you're doing the same things again and again at a different stage, at a different scale every day. Like, in version 1, you're probably like, oh, I want to build this. One engineer, one designer, just deploy the shit, go to the users. But if you have like 100 million users and like 200 engineers and like 50 designers, it's a completely different thing. You just don't even have any say in anything. Like, hey guys, like, I think the problem is this. And people are not coming back to Pinterest. So just redesign this entire experience. 50 people prototyping for the next 6 months, 100 people building code for the next 1 year, and deploying takes another 3 months or something. So everything gets slow. So it's you, since you are the CEO, you are the co-founder of the company, how fast do you want to deploy? How fast do you want to learn? And how fast can you do it? This is what I think a company should like all, like a co-founder's job is to like keep this loop running. A CEO's job is like, has my loop any loopholes? Like, is there any break in the loop? How can I fix it? Do I need to get a new guy? Is there bad blood in the team? Or is there problem like money? Or like, can we throw money at the problem? And so that's something I learned by watching Ben closely every day. There's a very interesting book that I read a while back, which I feel— I don't know if a few people have read. This guy Nir wrote the book called Hooked. It kind of explains every product in different things. I forgot the 4 different quadrants. But one is investment, and what do you get out of the product, and does the user find value if he does? It becomes a habit in their lives. And like how he explains how every successful product like Facebook or Pinterest or anything like becomes a habit over time. Like the first thing you get out of bed, some people are just like browsing Instagram or Pinterest because it's a habit for them. So I think it's an interesting read. I personally don't think you can build a product like, I'm gonna build a product which is gonna fall in the 4 quadrants. It's the other way. You build a product and like it happens to fall in the 4. So I would say just like read it and just forget it. Yeah, I think that's it. I'm done with that. I just want to leave a quote. Like the more like I mature, I always feel like, shit, I should have done that, should have done this. So as a people or the future entrepreneurs, I don't want you to have any doubt about anything. Just figure out what is the cheapest way I can build it and the cheapest way I can learn from that experience. And then move on.

Thank you.