Michael Kaminsky explains how to give executives the single answer they want without lying about uncertain data. Use probabilistic ranges with profit estimates instead of vague directionals, and show the financial risk of being wrong. Honest uncertainty beats false certainty.
Chapters
One of the things that sometimes people raise when we're talking about this testing program and we're talking about uncertainty is, my executive team hates uncertainty. I have to come in with an answer. If I bring in these ranges, they're going to be really frustrated with me. Do you have advice to give to a person who like has that concern?
I think it's about the communication of it. We often hear the words directional and directional can mean anything, right? It can just mean it was 0.01% higher, or it can mean that I feel like it's true. I think that kind of language does push the idea that we just don't know the answer. Maybe that's a true reflection of where we are and we need to test differently and we don't really get a clear result. You usually have some view from any kind of, let's say, user-level or geo-level test of the ranges that you can be quite confident in. And you can say, well, look, it's somewhere between here and here. And here's the probabilities of it landing outside of those ranges. And I think that alone does help you make decisions. If you can provide it with that level of transparency, 'Even if it is a wide range, this is profitable, this is not profitable. This could be very profitable. This could be slightly negative. What's the risk of making the wrong choice here versus making the right choice here?' And I know that there's a frustration when it's a very wide range. There's a 50% chance that it might be worth it and maybe not. But if that's the reality, I think that's what you have to work with at that point.
