The least-disruptive ways to measure incrementality (and the tradeoffs)

What are the types of measurement techniques that are less disruptive, that sort of keep you a little further out of the way of the marketers? And then every measurement technique comes with trade-offs. So what are those trade-offs and how do you think about navigating that? To go to one extreme, the most disruptive is to hold out half a country or to turn off an entire channel, something along those lines. That could be a user-level test that requires a 50% holdout. That's my cleanest read of what the true incremental impact, and that's something I really want to know. But is keeping a 50% control going to be more valuable than keeping a 10% control from an analytics point of view? Yes, but maybe not enough to change the decisions that you would be making. There's geo tests, but geo tests often require restarting a campaign. They require changing something that people may believe is already working very effectively. So you have to balance that need. I know that geo tests are great for their transparency. They are very easy to explain to someone. I turn this place off and let's see what happens. And so they really build trust really effectively. Geotests, I think, can be disruptive too. The less disruptive end of things, I would say, is a media mix model or some kind of regression where you're saying, let's see what happened and how does that impact our ability to predict these things. It is a less disruptive way of measuring. It can be disruptive again when you say, well, I need to feed this model with some volatility and I need to pulse and I need to do these things. That's not the way that I would necessarily be operating if measurement was not my goal. Some channels do lend themselves very well to kind of platform-level, user-level holdouts, things like that, they're not very disruptive. So it does depend on the kind of functionality available per channel.