Shopify CEO Tobi Lutke explains Goodhart’s law and why he doesn’t like KPIs or OKRs

Now Goodhart's law is real. The moment you create, like the moment a metric becomes a goal, it's no longer a useful metric. Right? That's, that's, I think more or less a precise wording. Why? Because no metric by itself, proxy is a complete heuristic for a complex business. Because this is not complex. There's a million of different tensions in a company and you can't all keep them in harmony by optimizing for one thing. So it's true that we don't have KPIs and we don't have at least OKRs in the sort of Silicon Valley sense, but we are extremely data informed. We invested enormous amounts of money and money and time into systems that give us basically everything on our fingertips. What Shopify attempts to do is to. And in a lot of different places, this is one of them. But also in products, it's just not overfitting for the quantifiable because I think there is like, again, this is one of those things. Everyone competes for everything that's highly quantifiable because it's like. Because it's like it really, it's kind of fun. It's like a game. You tweak a number and like 0.1 more is better than 0.1 less. And so that's an immediate gratification thing. But I just think the overlap of most valuable things you can do with a product and the things that happen to be fully quantifiable is maybe 20%, which leaves 80% of a value space unaddressable by the people who only look at quantifiable things. So what actually happens is Shopify is comfortable with unquantifiable things such as tastes, quality, passion, love, hate. It's like the strong emotions that people have. The sort of deep satisfaction that a craftsperson feels when they've done a job well is actually better Proxy, if you allow it to be, than do the unit test pass. The unit tests might not pass and the unit tests will pass 15 minutes later because we fix them. Or just the one or two things so they support us. The growth numbers we're looking for, we have systems that tell us exactly if something goes the wrong way. There's an extremely sophisticated rollout system in Shopify that forever holds holdouts and correlates everything with everything in every experiment and so on. But we think about it as a cockpit for a pilot and the decisions are still made by pilots and we think this leads to better results. Like there's plenty of AB testing tools and all these kind of things for commerce. And it's, of course, like, really important to figure out what your conversion rates are. But are you representing your brand is an unquantifiable question. Are you proud of the thing that you have built? Do you feel it's your own? Right. And so I think there needs to be more acceptance in businesses of unquantifiable things. So it's a bit of a more nuanced conversation, I suppose, maybe slightly less good on a fortune cookies saying, like, Shopify doesn't do OKRs, or it doesn't do metrics and so on. But it's actually just because the metrics take a support function where we often defer to just more, you know, sometimes a little bit emotional. But, like, generally less quantifiable things.