Michael Kaminsky explains Marketing Mix Modeling, a process that helps brands understand which marketing channels actually drive sales. MMM untangles overlapping campaigns to show what's working, though interpreting the data remains complex.
If you were to, you know, imagine you're on a sales pitch, you know, talking to a brand that's in the latter versus a brand that is much more mature, maybe differences in the way you talk about how mmm actually works. Like if I wanted to just peek under the hood and be like, mike, how does this work, dude? How would you explain that?
This is a really good question. And honestly, like, I'm going to give you what I often use, but I'm not sure this is the best explanation. So y' all can give me feedback on it. Okay, so one way that I talk about it is I say like, look, let's imagine the e easiest case, right? And this is an analysis that you could do in Excel. So you're spending money like normal, and then one month you decide you're going to invest a bunch of money in a television campaign. So in March, you know, we're going to keep spending all of our other channels normal. In March we're going to spend a bunch of money on tv and then at the end of March, the TV campaign is going to come down. How would we analyze the impact of tv? Well, we would plug our sales numbers into an Excel file and we would look at the start of March and the end of March and we'd say like, okay, did we see sales go up around that time? And then do we see them come back down at some point after March? And that's going to tell us a little bit about what was the impact of that TV spend. You can imagine doing that report in Excel and then like putting it on a PowerPoint deck and presenting it to the board. It's pretty easy to understand.
Yeah.
And so it's like, okay, let's imagine that we did that in March and then we did some radio in April. And so it's like, okay, now it's a little bit more complicated, but you can still imagine doing that in Excel. And then you're like, okay, well now we get to the real world where it's like, actually we have a bunch of different marketing channels, but they are all actually moving around some. On some days we spend relatively a little bit more on tv and on some days we launch in radio and then come down. And then on some days we spend, you know, we accidentally leave Facebook uncapped and it spends a bunch of money and then we pull it back down. And so what you can do is you can imagine like using machine learning to look back at those patterns on a much more fine grained basis than what you would do with in Excel in the March TV campaign, but still find those relationships and still do that analysis. On days when you spend relatively a little bit more on Facebook, do we get relatively a little bit more revenue on that day and subsequent days? And that's what the MMM is doing under the hood is it's looking for those patterns. When you say, okay, controlling for everything else that's going on, when we spend relatively a little bit money, more money in some channel or relatively a little bit less money in some channel, when is the downstream impact of that. And then we can use the statistics and the machine learning to stitch those insights together over time. And that's really what the goal of MMM is. How was that?
That was, that was pretty good.
Okay, I feel. Okay, let me know if what the follow up questions are and we'll see if we can, we can get even further.
Here are the follow up questions. I think the follow up questions are, you know, for brands that have more complexity, right. Because that's easy to understand. But then it's like, okay, we've got way more going on than testing one thing in one month and then another thing in another month. You know, we've got Sephora pushing a promotion in March. We're doing out of home same month. Sephora's reporting us numbers back in next month. You know what I mean? It's like, it's complex. So then the question becomes, all right, well, how does this actually work when the cadence is what I just described, you know, and then the objection is like, is this actually going to help my team? You know, is the cost going to be ROI positive? The cost of recast, the cost of doing this, is that going to be ROI positive? And how do you overcome those objections and answer those types of questions?
Yeah, so really good question. So the first thing, and this is often informative for helping people think about this is let's imagine the worst case scenario, right? Let's imagine that we have two channels that are perfectly correlated, which is to say that they, you know, move up and down at the same time. So in this TV example, instead of saying, hey, we're going to go, we're going to have the march of TV, it's actually the march of TV and radio. @ the beginning of March we spend up in both TV and radio and at the end of March we spend down on both TV and radio, we are not going to be able to know which, whether it is TV or radio that is having an impact. The model will not be able to differentiate between those two programs. So that is a fundamental limitation of what we're talking about here. To the extent that there are two totally new things that are happening at the same time, no statistical model will be able to differentiate that. There's no identification strategy there, which is what a statistician would say. And so there are fundamental limitations here in general that is not happening where they're perfectly correlated right there. If you're doing a Sephora in store thing, yes, that's happening. But maybe the radio thing actually started, you know, a week or two before and then goes a week or two longer. And now there are differences the model can use to start picking up. Okay, well, how much could be due to radio and how much is due to Sephora? Due to the timing of when the sales happen and when the marketing activity is happening, there will likely be uncertainty there. And depending on the quality of the mmm, the quality of the model, we might, you know, have more or less uncertainty in differentiating between what is what. But that's sort of the best that we can do in that scenario. And so I would say, you know, to someone asking these questions, it's like, yes, there will be uncertainty. However, getting some signal is probably better than no signal. And if we're talking about Sephora sales, like, what other option do you have? And so if we can get some signal that can help us make better decisions, even if it's not the perfect decision. And so that's a way to start thinking about, okay, is it this going to be worth it for us or not? I would definitely say that, like, MMM is not worth it for every brand. And you should absolutely think through, like, what is the ROI on these insights that we're actually getting in terms of being able to action off of them. But just because things are happening at once doesn't mean that it's impossible to get a signal. And also I would push people to say, like, is it better to get some signal rather than no signal? And then what's the value of that? You know, perfection is impossible. We're never going to get there. But if we can, you know, make better bets in the long run, that should pay off for us as a business.
The Continuous Testing Loop That Separates Great Marketing Teams From Average Ones
