Measuring Marketing Incrementality and Budget Optimization: Michael Kaminsky, Founder of Recast

Hello and welcome back to the show. This is Better Advertising with betterams and I'm your host, Justin Nuckols. I manage ad strategy and delivery for some of our clients here@betterams. But I also have the privilege of talking with leaders in the industry who are building brands or managing advertising strategies. Today's conversation falls somewhere in between as I'm welcoming on Michael Kaminski from Recast. As a data scientist, Michael's company helps brands with marketing mix modeling, which essentially helps predict where your next dollar spen will be most effective. I'll leave the Recast elevator pitch at that and let the man himself tell us more. Let's welcome Michael.

Thanks for having me, Justin. I'm excited to be here, excited to.

Have this conversation going. We got connected over email a couple weeks back hearing a little bit about your company named Recast doing a lot of kind of data modeling Fusion. I'm probably getting that wrong, but I'll let you give kind of the background of your company and who you guys are.

Yeah, thanks. So I work, I co founded this company called Recast. We do modern marketing mix modeling. Basically what that means is we help marketers and brands measure the effectiveness of their marketing spend and then use that to do budget optimization and planning. But really my background is in marketing analytics, sort of more broadly. Before starting Recast, I ran the data science and analytics team at a company called Harry's. They make men's grooming brand, they make their men's grooming brand, they make razors and other men's grooming products. And I was along for the ride at that business as they went from being D2C harry's.com only to expanding into Amazon and Walmart and Target. And so I spent a lot of time thinking about how to do marketing measurement, how to think about marketing efficiency in these complex businesses that have multiple different distribution channels. And I'm, I'm really excited to be here today.

Yeah, that's a very unique position to be in where you are, or you were rather sitting in the seat of a lot of the people you were helping, where you were overseeing or helping a team oversee five, six to 10 marketing channels. And how do we best optimize that? So you sat in the driver's seat, felt those headaches, and you built a tool to kind of help people answer those questions. Now, so as it pertains to Amazon, how much did you see Harry's while your time there get into Amazon? Or how much of the headaches of Amazon did you have to deal with at that time.

So while I was there, not actually that much. I think Harry's was sort of fairly late getting into Amazon. There was a lot of questions internally in the business at the time about like cannibalization between the D2C business at Harry's.com and Amazon and how we wanted to think about leveraging that channel to the, to the best that it could be. What I can say though is that now we work with a lot of brands that are sort of either dual channel, where they sell a lot of their goods on both Amazon and their, their d2c.com Shopify site or omnichannel, where Amazon is a part of the mix alongside other retailers like Target, Walmart or Grocery Drug, et cetera. And so have been spending a lot of time over the last like five or six years really thinking about those problems in terms of how do we optimize our marketing mix in a way that moves the whole business forward, not just the D2C site, which, or the Shopify site which is easier to measure, but for a lot of the brands that we work with, it's actually like sort of only a small fraction of the total business where they'll have a huge Amazon presence, some amount of B2C and then a few other retail distribution as well. And that world can be really complex because your last touch attribution report from Google Analytics isn't really telling you about what, what marketing is actually driving your, your Amazon sales. And so it's, it's a really interesting and I think an exciting position to be in to start to think about. Okay, well, how do we pull these pieces together in order to take the evidence that we have about what marketing is working, but then make decisions that's going to work for the whole holistic business, not just the things that are easy to measure on the dot com Shopify website?

Yeah, that's a question that we often face here even in Amazon is, is what you're doing on social media impacting what we're doing on Amazon or is it streaming TV that's driving more Amazon? So that's a question certainly within the world of Amazon that we're all trying to answer a little bit more clearly. So what kind of questions do you guys help brands face and answer at Recast?

Yeah, so I mean the most important question that I think marketers face or that if you're not thinking about this now that you probably should be thinking about is what is the incremental return of an additional dollar invested in a marketing channel and that might be paid social or it might be YouTube or streaming TV or even Amazon ads themselves. And the idea here is this, this idea of incrementality, which is to say the causal relationship between that marketing activity and additional sales. Just because someone saw an ad or clicked on an ad doesn't mean that that ad necessarily drove that purchase. Right. I'm sure you can imagine this happening at Amazon. If someone is searching for, and you run an ad and they click on it and you pay Amazon for that, were they going to purchase anyway? Right. That's the big question that, that marketers need to be thinking about and need to answer. And they want to know, okay, if we run this YouTube ad and it has an ROI of 1.5X, if you just look at Shopify, if there's any, is there a halo effect on Amazon sales as well? Should we be thinking about that as we're, as we're optimizing our marketing budget? Those are the sorts of questions that marketers have that Recast really helps them answer. And then taking that information and doing budget optimization. Look, let's take money out of this channel that maybe looked really good on Last Touch Google Analytics reporting, but actually isn't as good as these other channels. Once you take into account the true incrementality and the multichannel effects of that marketing spend.

That is fascinating. That's a, that's something I can't even think of how you begin to figure out. So, so where do you even start with, with understanding that. I mean, checking out your website, which we're going to link here in the show notes for everybody. Recast.com or get recast.

Get recast.com. there you go. We couldn't. We're a startup. We can't afford the one word. The one, the one word domain name yet.

Yeah, get recast. Com. I mean, looking at a lot of the things you guys do, understanding when you're hitting diminishing returns and oversaturation. I mean, is all of this just data modeling? Just. I mean, I'm, I'm a data guy myself. I like to geek out on this kind of stuff. But is this all just modeling or how do you even begin to measure this incrementality and where the next dollar spent is best spent?

Yeah, great question. So I can talk about a little bit about how we do it at Recast, but then maybe we'll even take a step back and we can talk about how different brands at different stages think about doing this. Because the right way to measure this and the right approach. You want to take is going to vary depending on what size of company you are and how much you're spending and really what problems you're trying to solve. So at Recast, we do this thing called marketing mix modeling, which has actually been around for a very long time. So like cpg, companies like Pepsi and you know, Gillette have been doing this for 50 plus years probably. It's an old idea of using statistical models to find patterns in observational data that a brand might have. So you might look at your historical data and a time series of, hey, on this day we spent this much on this channel, and on that day we spent that much on this other channel. And then you look at the time series of sales and you use a statistical model to find the relationships there. When we tend to spend more on tv, controlling for everything else, it tends to yield X additional dollars of sales. And that's sort of the fundamental idea behind this idea of marketing mix modeling, which of course can be applied to any distribution channel. Right. There's no digital tracking. We don't rely on clicks or impressions or who viewed this and then what did they click on on the path to purchase? None of that is, is really important at all in this framework, but rather we're just looking at this aggregate data. So that's sort of the idea behind what we're doing at Recast is marketing mix modeling, but done in a way that's sort of much more automated than, than than what was possible in the past. But for brands that are sort of smaller and just getting started and starting to think about incrementality and how do we think about it? I generally recommend that you sort of start with thinking about experiments and experimentation. Right. And so this is a place where you can actually, I think, and I'm curious to get your perspective on this, Justin, if you've seen this executed well, how can we run experiments where we run ads for some subset of people and don't run ads for some other subset of people and then look at the relative lift of performance and sales between those two different groups. So one way that you might imagine doing this is look, let's run Amazon ads for the eastern half of the country and not the western half of the country and then look at how did our sales change when we either turned on ads for that half of the country or turned off ads for that half for the other half of the country. And that can give you a sense of true incrementality and tell you, are people just, are people just clicking on the ads and they were going to purchase anyway and so that's an unnecessary expense for the business or are those ads actually driving true incremental revenue for us and how much? And that's I think a good way for people to start from thinking about how they can bring this idea of incrementality into their business. Especially for channels like Amazon where we have limited insight into what people might have seen advertising wise before they actually got to Amazon. Com.

Yeah, yeah, no that's, that's a fascinating approach is like isolating the impact of every variable kind of. In short is what that sounds like is, is we're taking if I do this or if I spend more here, if I cut budget from this ad type, what will happen to the rest of my metrics, sales, so on and so forth.

That's exactly right. And I'm curious Justin, like have you seen the clients that you're working with? Have they done this sort of experimentation? Have they rolled it out? Have they had any success with doing this sort of work?

Yes, we all, we are always big fans of testing but the, the isolation of variables is always a hard point to kind of remove from the test. Is we the most simplest example. Right. Is if we are going very aggressive on a new ad type that maybe a brand hasn't tried before. Like we're, we're going really aggressive at sponsored brand. Well, there's a bunch of variables thrown into there as well. Was it the creative image we use? Was it the headline we used? Was it the product shown in the headline ad that we use? Like there's so many variables to try to isolate and obviously more data helps. I'm sure that's, that's no surprise to you. Like we could split test a bunch of sponsor brand ads to see what the best looking ad performs. But yeah, I mean isolating variables is always, is always tricky and I think the easiest way to your, to your example of audiences is probably DSP which kind of segues, segues into my next question here. Whereas DSP we can get very granular with our segmentation of these audiences, excluding others. So I'm curious, how much experience do you guys have at Recast with working with Amazon sellers that use not just PPC ads but DSP ads in addition to a long list of other ad types, I'm sure.

Yeah, so, so we work with a number that do that sell a lot of on Amazon and they advertise in a lot of different channels, everything from display via DSP to YouTube to streaming TV et cetera. And they're often all sort of trying to think about, like, again, what is the true incrementality of these different types of ads? One of the things that we there. So there's different ways for people to think about either using Recast to answer those questions or even running experiments to do it. One easy thing to do is just to, like, turn off ads for a certain channel for some fixed amount of time. So we might be spending, I don't know, $5,000 a day on some marketing channel. And then what we're going to do is for three weeks, we're just going to take that to zero, and then we're going to see what happens. And we can look@.com, right, we can see how does sales change. But really importantly, we can also look at a time series of our Amazon sales and say, like, okay, before we turn this channel off, we were getting $200,000 a day in Amazon sales. And then we turn that channel off and it went down to $175,000 a day. And that gives us some insight into, okay, well, that channel was probably driving about $25,000 a day worth of sales for us. And then we can plug that in to figure out, okay, was that worth it? Once we take into account all of the different distribution channels that we have, that's going to be a good way to think about what the true incremental ROI of that channel is. And so one thing that I'm often trying to convince brands to do is to think more about how can we take these, make these bets, take these swings, run these experiments in order to get more signal. And you don't always have to like, turn advertising off, right? You can also turn advertising up and say, like, we're going to double our spend for three weeks into this channel and then see what that does to our overall sales. And that can start to give you a way to start to think about, okay, what is the incremental return of this additional investment when we really vary the spend. And that's a really great place to get started is just doing that sort of one channel at a time when it makes sense. I mean, of course you can't do that around like Black Friday. You need to be in some, like, stable zone in order to be able to run these experiences, experiments. But it's a really good practice to build up. And then once you're doing a fair amount of that, you can layer on some of the more advanced statistical techniques, like what we do at Recast with marketing mix modeling to sort of statistically help us put all of these different pieces together when we have 15, 20 different marketing channels that are all moving around and we want to get ongoing signal as opposed to just these one off experimental reads.

I want to go back to that Black Friday point you made as we're coming up on Black Friday is, is how do you sort through all the noise? As I know from the advertiser's perspective, a lot of brands, they like to try things very frequently, right? We like to try this deal and discount this coupon. Let's actually lower our price $10. How do you run a test with these brands, kind of isolating out all these variables and noise that is impacting results?

This is a good question. And I think it's really tough. It's tough to do, right? Especially, especially with Black Friday in particular, where there's so much going on because it depends on what your competitor brands are doing and what you're doing and like the natural seasonality and how hot your product is this gifting season or whatever. So there's lots of confusion there that can make this analysis really tough. What I recommend to brands, though, is to try to be as deliberate and thoughtful as they can about setting up tests to maximize learnings within the constraints of the business. Right. So I obviously rec. I'm a scientist at heart, so, like, I love just like learning stuff and running experiments. It's not always possible to run an experiment, right? You have to run the business first. You have to make sure that you're getting a good return for your business above all else. But once you meet those constraints, you do want to be, I think, to the extent possible, thoughtful about trying to be able to isolate these different changes.

Right.

So if we're going to, if we want to experiment with like different types of offers, can we do it in a way such that we can get good reads on what is working better or worse? And that might mean showing some offers to just some subsets of people. So that way we can compare afterwards and say, okay, group A saw this offer and group B saw this offer, and this is how the conversion rate differed and what the total profit generated was. And that will give us some information that we can use next month or next year when we're coming back into this season. I think the bad thing to do is just to like run a hodgepodge.

As I'm sure you've come across. A lot of advertisers and brand owners who say, if it, if it hits 10 clicks and no sales, I want to not target that audience or I don't want to target that, that keyword anymore. AMC is starting to show us like how many clicks across, not just one target, but across ad types. Dsp, ppc, and that's sponsored product, Sponsored brand, excuse me, sponsored display and showing. It takes various touch points across all these ad types to lead to a sale. So it's almost like a condensed version of that mixed media modeling, but within the lens of just Amazon advertising. So there's, there's really like. Go ahead.

No, I mean, I was just gonna say, like, what I would really love to see in this toolbox is experimentation tools. I think Meta slash, Facebook has been rolling out some really interesting tools over the last couple of years to help brands run these like Lyft experiments. So effectively the way that this works is if you're spending enough money on the platform, you can run a lift test. And what Meta will do on your behalf is they will take people who are in your potential audience and split them into two groups. One group that will be eligible to see your ads and one group that will not be eligible to see your ads. And then you'll run this experiment for a month or a couple weeks or whatever. And then at the end they can produce a report that says, hey, look, among the group that saw your ads, we saw an increase conversion rate or an increase total number of conversions of X compared to the group that was hold out, that was held out. And the group that was held out still might make purchases. Right? And so the important thing to do is to look at the difference between the ones that saw the ads and the ones that didn't in order to get a read on that true incremental effect. I think, I honestly think that Meta's technology and software here is really good. And I think that that sort of tooling is what I would really love to see from some of these retail media channels like Amazon, is to give marketers the ability to run that type of true incremental lift experimentation in order to get a really solid read on what is the actual ROI of spending on, on advertising on this platform. Because I think a lot of brands smartly have this question of like, look, I'm spending this money on Amazon ad, but if people just like scroll down, they'll see my product. So like, how valuable is that ad? Am I actually getting a good return on investment for that ad or not? And that's like the question that I think ends up being really important. So I'd love to see Amazon and other retail media media channels start to build in that sort of experimentation, tooling that allow brands and marketers to get a really crisp read on exactly that question.

Had had we had this conversation one week earlier, I could have put in that request to the Amazon executives out at Amazon unboxed and we could have, could have seen that come in development here soon. But like I was saying, I think DSP is probably the best place for advertisers today to run these kinds of tests and answer questions around the incremental impact of Amazon advertising. But within ppc, you're totally right. We don't have that opportunity to say, hey, these products, I don't want to show up for PPC ads at least consecutive. You'd have to go all or nothing with let's turn all ads on, turn all ads off for these products, these ad types, whatever you're trying to isolate and test. So it'd be really great to get that testing aspect within PPC soon.

With DSP and PPC and Amazon, do you have the ability to like segment by geographic region? So could you say like, hey look, we're going to turn off, we're going to leave our ads on everywhere except for Florida and then we're just going to turn off Florida. Is that a, is that a thing that you can actually set up and run in the platform that way?

Short answer, yes within dsp, very easy. No within ppc. Unless, because I know some sellers going to say, well, you could do it if you had SKUs that were specific to certain regions. And yes, if you were only advertising, maybe FBM fulfilled by merchant ASINs that were fulfilled maybe on the west coast or east coast, if you organize your catalog that way, so be it. You could run that test. But I don't think anybody that I've ever met has SKUs specific to regions. So yeah, nothing native within the.

That makes it challenging.

Yeah, nothing native within the PPC ad console will give us that ability to segment by geography.

Yeah, okay, so that, that's really fascinating. I'm obviously not an expert in how to manage these ads on Amazon, but I think with, at least with dsp, like what I would encourage brands to do is, is do something along those lines, like let's turn off ads in a couple of different states or a couple of different zip codes or again, whatever makes sense for them. And then see how much do sales drop in those, in those geographic regions and that can you, if you can compare the ones that kept ads on and the ones that had the ads turned off, that can give you a good read on incrementality at least for the ad types that are available to do that type of experimentation. And I think that's a really cool analysis to be able to go run in order to prove the true incremental lift of showing those ads to those types of customers.

Yeah. And I think question for you is how would you run something like that for a more regional based brand? Right. Like if I'm selling winter products and that's, that's obviously going to do really, really well here soon in the Northeast. I'm out here in Phoenix, Arizona. So it's, it's not.

Go Devils.

Yeah, go go Sun Devils. Michael here, fellow ASU grad. So yeah, we're, we're not looking for our cold gear until like January, February, if ever. So how do you kind of segment test when your product itself is very regional?

Yeah, really good question. So the way that I would do it is I would pick like two regions like New York and Boston or New York and New Jersey just to like throw some things out there. Again, you could even do like New York and Philadelphia or Philadelphia and Boston versus New York and New Jersey and then pick. So you, you want to pick regions that are like relatively comparable if you have a product that is going to vary in sales by region in order to run an experiment that way. And so in that world you would maybe keep Boston and Philadelphia on getting ads and you would turn off New York and New Jersey or vice versa or whatever. Again, you know, you have to be careful about. You don't want to put your business at risk here if those, if that, if all of your sales are coming in one month, like that's really risky to do.

Yeah.

But if you can. Right. And you can pick some region that looks like these other regions and you can experiment with. Look, you know, in this region we're going to double our ad spend, right. We're going to increase our bids. So we're getting way more touch points. In this region we're going to keep it flat and in this region we're going to cut it in half or turn it off. Then you can look at the total sales across those three regions like before and after, after the change and get an estimate for what the lift is and what the lift is at those different levels of investment in, in, at, in Amazon. And I think that that's like an experiment that most brands should be able to run. Again, if you have the sort of operational capacity to go put this into place, that can give you a lot of insight as to the true effectiveness of that spend Absolutely.

I mean, if anything, this conversation is showing me like as as much of a numbers guy as I claim to be. Like, you need to talk to a data scientist. So shout, shout out, recast. If you guys are having these questions, please hit up Michael and talk, talk with him.

I love to nerd out about this stuff, so reach out anytime.

Next question here for you man is as we know, these, these big ranking pushes or big market share pushes, right? If we know we want to make a big push on a certain ad type or a big incremental change, how quickly do you come back and remeasure the impact? Right? Like if we're, we're seeing the next dollar spent is best spent over here and a brand makes that decision and says, great, let's go forward with that. Six months down the line, 12 months down the line, do they rerun this test and say, okay, things have changed, actually we should move budgets over to this social media YouTube ads.

It's a really good question. No easy answer here unfortunately, because it depends as I'm sure everyone expected that I was going to say. So I think like this is a place where I actually think that the marketers and the channel managers and the channel experts really need to weigh in in terms of thinking about, okay, what are the dynamics of this channel, how does it work and when is a good time for us to remeasure. There are some channels that I think are generally more consistent over time, right? Things like linear television and radio, like they tend to operate the same way. The CPMs don't tend to move around nearly as much as some other channels. And so you could sometimes have more confidence, right, that if you get a good read six months later, that read might still apply. Other marketing channels, especially those that are like dynamic and bid based. So any sort of market advertising, marketplace type channel, social media like Facebook or Meta, I would imagine Amazon works the same way, have very different dynamics, at which point you probably need to be measuring them fairly regularly because what your competitors are doing or what other people who are in the market who are not even direct competitors are doing can have a really large effect on your return for your advertising investment. Additionally, there are other things on the platform like the number of reviews that you have and the popularity of your product. Again, I'm not an expert in this world, but these are things that I imagine impact your performance that might change how valuable a click on a, on a PPC ad might be, right? So like again, if you are a really popular product, it might be the case that you don't need to pay as much for advertising. If you're just getting started and you need to get those reviews and you need to climb the rankings, then advertising might be very valuable to you. And so as your position there changes, you might want to think about retesting and reevaluating the true performance of that marketing. Because what was working six months ago because of where you were as a business or seasonality or the dynamics of the platform might no longer be true today. But again, it sort of requires an expert on the channel to. To be able to reason through those different aspects and where you are as a brand and where your product is in order to be able to make those decisions.

Yeah, no, that. That makes sen. It takes the expert in the seat of the marketplace to know, like, when those variables have changed enough and we're in a new stage of the lifestyle, the brand new stage of. Yeah. Of growth. Well, awesome, man. Let's see here. Any other questions I got for you? I think that's. That's about it for my questions. How. How much more can I plug here for recast? Anything else I can shout out for you on. On the website? Definitely want to drop in the link.

So I would say, like, look, yeah, the link is great. People can find me on LinkedIn. Michael Kaminsky just searched for me. I put out a ton of content on marketing measurement, marketing, mixed modeling in particular. You can also find me on Twitter, Ike Kaminsky. I love talking about marketing measurement and whether that's experimentation or just like thinking through incrementality and where should we be investing and what new channels are interesting to go and test. Those are the things that I'm really passionate about. So I would definitely encourage any listener to reach out with questions you have or thoughts you have or interesting experiments that you've run and would love to hear about them and talk about them.

Yeah. Please reach out to Mike. If you guys have questions or found this episode fascinating as I did, please reach out and pick his brain a little bit more. Thank you so much, Mike, for being here today. We'll chat soon.

No, absolutely. Thanks for having me, Justin.

Thank you.