Tom Vladek, a statistician and Recast co-founder, walks through the core challenges of media mix modeling and how to avoid common pitfalls. He covers why MMM is harder than it looks, what makes a model reliable, and the specific steps to build one that actually works.
Have you ever wondered what makes the difference between a good and bad statistical model? Or how to avoid the pitfalls that most modelers fall into when building statistical models for real world problems? You're in the right place. I'm Tom Vladek, statistician, marketing scientist and co founder of Recast. I'm here to show you the secrets and science behind building complex and reliable models in the context of media mix models Built on top of Recast this is how to build a media mix model that won't destroy your marketing program. Presented by Recast Episode one To start things off, let's face the facts. Media mix modeling is hard. It's really easy to throw your data into an algorithm and get results back, but more often than not, this will lead you astray. We've seen it hundreds of times. A brand wants to try this hot new thing and believes they can just dump their data into a black box algorithm or simple regression and get results on the other side. The fact is, MMM is incredibly challenging. We like to say it's harder to imagine a tougher statistical problem than mmm. It's a small data low signal problem where millions of dollars are on the line. If you've been tasked with building an MMM and want to generate results that are actually useful, you need to take the building process seriously for context. At Recast, we've built an MMM SaaS platform that helps brands develop, deploy and use media mix models efficiently. We have spent a ton of time thinking about how to automate and develop the best practices for developing a media mix model, and we want to share those findings with you. In the past, MMM modelers would do a ton of offline analysis in R scripts or Jupyter notebooks, and then eventually they'd compile their results into a PowerPoint deck and share those out. But here's the thing. If you're only building a one off mmm, it's really easy to bake in tons of cheats where you think you're generating sensible and actionable results, but really you've walked down the wrong path. We've heard of people doing really indefensible practices like freezing coefficients. This would mean presenting coefficients from two models as if they came from one, or even simply overwriting results they didn't like. Since Recast is designed to be efficient and automated as well as ethical, those practices simply wouldn't work for us. So we had to take the state of the art in statistical model development and apply it to mmm. In this video series, we're going to walk you through our model building best practices that brands have used to get the most out of their MMM and avoid the pitfalls that many brands fall into when building an MMM and using it. Here's what we're going to cover in the following first, choosing or developing a model Second, model configuration or prior setting based on background and expert knowledge Third, the configuration compatibility check Fourth, the parameter recovery check Fifth, the stability and robustness checks and finally, sixth, the out of sample predictive accuracy. We'll dedicate a video lesson to each of these steps so that you can follow along as you're building your own models. Now remember, we'll be explaining these in the context of the Recast platform, but each of these is generally applicable to statistical models in general, so you can use them for your own work. You'll just have to write a bit of code.
