In this persuasive, conversational video Michael Kaminsky explains why marketing performance dashboards are unreliable guides for budget decisions. He opens with a vivid illustration of conflicting metrics—ad platform ROAS, GA4, lift tests, Shopify revenue—showing how each tool offers a different “shadow” of reality. Kaminsky then invokes Plato’s Allegory of the Cave, likening dashboards to shadows on a wall that marketers mistake for truth. He defines the true metric marketers need: incrementality, the causal lift that indicates sales caused by marketing, not merely associated with it. Kaminsky argues that no dashboard can directly observe incrementality because it requires a counterfactual reality where the campaign didn’t run. Consequently, dashboards provide estimates that can be misleading. To move beyond shadows, he recommends concrete experimental methods: geographic holdout tests, turning off channels, and forecasting with subsequent validation. He acknowledges the psychological discomfort of discovering over‑investment, noting that many teams avoid testing because it threatens comfortable narratives. Finally, he stresses that marketers who regularly test and validate their measurements make better budget decisions over time, not because they have superior tools, but because they refuse to accept shadows as reality.
Chapters
The ad platform says 4.7x ROAS. GA4 says 3.2x. The lift test says 2.1x. Your Shopify dashboard says revenue is up 15%. GA4 disagrees. Your boss says it's all wrong. Every marketing performance dashboard gives you a number. They all look confident, yet none of them agree with each other. So you pick one, or you average them, or you go with the one that matches what you already believe, and then you make a budget decision. Because the meeting with your boss is in an hour and you need to make a call. That moment where every dashboard is shouting an answer and you still don't know what's true is related to one of the oldest problems in human knowledge. And a Greek philosopher described it perfectly 2,400 years ago. In the allegory of the cave, Plato describes a group of people who'd spent their entire lives chained inside a cave facing a blank wall. Behind them, a fire casts a shadow of objects onto that wall. The prisoners watch the shadows move. They give them names. They argue about which shadow will appear next. To them, the shadows are reality. It's all they've ever seen. It's all they've ever known. Now think about your marketing stack. Every tool you use, your ad platform, your analytics suite, your attribution model is showing you a shadow. It's a projection of something real, but it's not the real thing. That real thing is whether your marketing actually caused a sale. That's called incrementality. Not who converted after seeing an ad, but who converted because of it. Incrementality is the one number every marketer needs to know. The only problem is that no dashboard can show it to them directly. That's because measuring the true incrementality would require seeing two versions of reality at the same time. One where you ran a campaign and one where you didn't. Then comparing the difference. You only ever get to live in one of these realities, so every measurement tool is doing its best to estimate what the other reality would have looked like. That estimate is the shadow on the wall. And here's what makes the shadows dangerous. They look like answers. They seem to us like reality. A dashboard that showed you nothing would be obviously useless. You'd ignore it. But a dashboard that shows you a clean 4.2x ROAS, well, that's convincing. You build plans around it. You defend it in meetings. You allocate real money against it. And the number alone can't tell you whether it's close to the truth or completely off. In Plato's story, one prisoner breaks free. He turns around. He sees the fire. Then he sees the objects casting shadows on the wall. He walks out of the cave and sees the actual world for the first time. It's painful. The sunlight hurts his eyes. But now he knows what the shadows actually were. In marketing, turning around means testing whether your measurement is really close to reality. This could mean running a geographic experiment, increasing spend in some markets, holding others constant, then measuring the difference. It could mean turning a channel off, going dark, and seeing if anything actually changes in your revenue. It could mean making a forecast based on what your model says would happen, then checking whether or not it comes true. These methods don't give you the final answer on incrementality. Nothing does, because incrementality lives in a universe we can't observe. But a well-designed experiment gets you closer. It tells you, for a specific channel at a specific point in time, whether the shadow on the wall is roughly the shape of true incrementality or something else entirely. That's the goal. Not perfect knowledge, but signals that help us make better decisions than a dashboard alone can provide. Most teams never do this. Not because they can't, but because they're afraid of what they'll find. If you test your best performing channel and the real incremental lift is half of what your dashboard says, well, now what? You have to walk into a room and tell your team you've been overinvesting. That's a hard conversation. It's a lot easier to just stay in the cave, so most people do. The shadows are comfortable, the numbers look right, and nobody asks whether they're real or not. Here's what Plato understood that applies directly to your marketing budget: the shadows aren't lies. They're real shadows of real things. Your marketing dashboard is measuring something. The question is whether what it's measuring is close enough to true incrementality to make good decisions with. You'll never see the real object directly. But you can do better than just accepting the shadows others make for you. You can run experiments. You can make forecasts. You can check their results. And each time you do, you get a little closer to the truth—for that channel, at that moment, with those data. The marketers who do this consistently, who keep turning around and checking, they make better decisions over time—not because they have better tools, but because they stopped mistaking the shadows for the real thing. And you can do it too. You just have to be brave and get up and leave the cave.
