How to measure offline marketing performance right | Krystina Rubino (Right Side Up) | Coffee Breaks

Foreign. We're two minutes in, which means it's time to start. So thanks for joining, everybody. I'm so, so excited for today's session. As you can tell, Christina is a delight and we're delighted to have her. So, preliminaries, this is the marketing measurement coffee break. It's brought to you by Recast. As always, we make the same promise to our attendees. And if you're listening to this later, we're going to make these 30 minutes super efficient. We're going to get directly technical content and there will be no pitching. We are thrilled to have Christina Rubino joining us. Christina is the general manager of the offline practice at Right side Up Agency. She's been a longtime partner to Recast, helping us and then honestly challenging us on everything that comes to offline marketing measurement. So we're going to jump right in. I will ask everyone on the call if you have questions, if you have thoughts, comments, drop them in the chat. We will be keeping our eye on that. Um, so as things come up, it's more fun if this is like a conversation where we're getting questions from the audience to like double click into stuff. So please do it. Um, and with that being said, I will stop sharing my screen so you can see at least Christina's beautiful face and my ugly mug. And we'll jump right into the questions. Christina, Coupon codes, vanity URLs, what are your thoughts? Um, a lot of people use these to measure offline media, obviously, but what do you think about?

I think that they are an incredibly useful tool when deployed correctly and when not used as the. The end all be all. So just thinking about the way that we engage with offline media and audio in particular. If you're listening to a podcast or you're listening to a radio ad, you are probably doing something else, right? And if you're the kind of person that's staring at your screen when you're listening at audio for like hours on end, we should probably study you because you're not the typical consumer behavior. Right? The typical consumer behavior is I'm taking a walk, I'm cooking, I'm doing, I'm doing something else with my hands while I am listening to audio. And so that's why you see, and that's why recast it. And tools like it are so valuable. Not like really, that we need to look at vanity URLs and look at promo codes as almost like the equivalent of a click from digital, but knowing that not everybody clicks on an ad because you Know, I don't know. Most people have clicked on a banner ad since AOL was relevant, right? So it's, it's. The idea is you're capturing that indirect activity that happens via some other mechanism, but you still do need to replicate the direct mechanism that you're achieving in digital with a click. And so that is, to me, that's the function that vanity URLs and promo codes play in that process. It's triangulation. Right. And like anything else, you, you need multiple tools in order to triangulate. So often what I advocate when people are using vanity URLs and promo codes, great indicator of somebody who's high intent and like gave enough of a crap to like listen, write down, type in whatever. But you combine that with some sort of other attribution. So you combine that with post purchase survey attribution. You combine that with pixel based or server side implementation and podcasts. And I am a very big fan. We've had different ways of saying this, but overlapping and redundant methodologies. Something at some point somewhere will fail or something at some point will tell you something that you're like, this doesn't feel right or it doesn't seem right, or my Shopify data is telling me something else. So it's better to have different points of measurement. That way you can kind of gut check everything and just make sure, like, is this all? If it's telling a similar story, great. If it's not, why? And can I dig into that more and understand it?

Yep, all of that makes sense. A bunch of stuff that I want to dig into here. So you said something that was sort of interesting, which is like, think about, you know, the coupon code or the, the vanity URL as a click, which I think makes sense in some contexts, but maybe not in others. Right. Where clicks are relatively easy and cheap and I think happen a lot. Whereas like going to a vanity URL seems, you know, a lot more difficult. How do you think about like educating stakeholders, right. Who maybe aren't as invested, like don't know the details of this. And it's like the CFO is looking at the report and trying to make sense of it. How do you educate that person on the right way to think about looking at this report that has, you know, the vanity URL last click or whatever.

Yeah, I mean really, it's like the tip of the spear. It's not the whole, it's not the whole spear or like tip of the iceberg. It's not the whole iceberg. Whatever. It's it is a signal that is telling you the direction in which to look. The example I commonly use with vanity URLs and promo codes is people wind up searching for that, right? So it isn't, it's an innate human behavior to search. Also, Google has made us all incredibly lazy. And so let's just say that I, I remember that I heard an ad for, you know, toothbrushes, and so I'm going to type in blah, blah, blah, toothbrush that is not going to be captured by a vanity URL or promo code, right? But when you go through to the site and you go through, let's just again say the checkout process. So you go through and when you are surveyed, you say, hey, I actually heard about you on a podcast. That is the point at which you're now attributing indirect activity that contextualizes vanity worlds and promo codes. I tell people, if you are only looking at direct activity, which is what we are trying to use vanity URLs and promo codes for, you should not do offline advertise because you are going to wind up missing anywhere between 40 and 80% of the activity. So if you think of it in the context of a multiplier, for every one person that actually gives enough of a damn to like, type in the vanity URL or go to your website and enter the promo code, there are anywhere between two to seven or eight people, depending on the medium, depending on the product, depending on the strength of your offer, the strength of that promo code and promotional activity. There are so many other people that you are missing their activity. And this is where it shows up in brand search. It shows up in direct. It shows up maybe in noting that, you know, your, your conversion rates week over week on Meta were much better than prior to launch. So that's the way to contextualize it. You should never be looking at just vanity URL or just promo code because it doesn't tell you the complete story. And then also promo codes leak. So like, you have to, it's a balance of like, don't make the promo code so great that everybody with a pulse is going to be like, yeah, I want to do that. You've got to make it just good enough for people to use it so that you have triangulation and then not kind of like see it leak to the coupon sites and stuff. But again, that's why it's important to have overlapping methodologies, because if you do see a leak, you could still triangulate performance because you're like, oh, this is what happened. These are the days it leaked. We swapped the promo code on the back end of the vanity URL, and now we know from this point forward it's actual data as opposed to the leak data.

I think that makes a lot of sense and it's really wise. I mean, we talk a lot about triangulation. I think one of the things that jumps out from what you're describing here, unfortunately. Right. Is that it's not. There's no, like, silver bullet here. It's not easy. Right.

I'd be on an island somewhere if there was a silver bullet, because I would have started a company and sold it for a bunch of money and I. You would never see me again. Yeah, absolutely. There is no silver bullet. And anybody trying to tell you they have one is full of it. So, like, there is no one defined way to do this. Like, how many. Honestly, Michael, like, how many clients do you guys see where it's like, okay, like, we don't really have a complete story. The data is telling us this. The data from Shopify are, you know, this and that. Like, it's always. I have never heard any one advertiser be like, we have it figured out.

I think, well, look, I. There are a lot of at least, like, partners that want to claim that they haven't figured out. Right. I know a lot of, like, vendors.

Give me an hour with them.

But advertising vendors, yeah, no, we have the method and it does the thing. It's the silver bullet. So I think it's important to, like, recognize and be honest with ourselves and like, our stakeholders that there isn't a silver bullet and there's parts of it that are going to be hard and, like, it's going to require thought and someone to, like, doing a custom analysis to. To figure out what's actually going on. We got a question, and I actually have no idea, like, what the answer to this is, but do you have a thought on, like, leaked coupon codes? How frequent is it? How big of a deal is it? You know, what is the proportion that people should be expecting from that?

Yeah, I mean, listen, it depends if your regular discount on your website is just free shipping and all of a sudden you have a 25% code out there in the wild somewhere, like, people are going to find it. Deal sites and coupon sites are constantly scraping retailer, website, and so what you have to is there's a couple of ways to combat it. One is doing no index, no follow on any of your vanity worlds. If you're not Doing that, please, like run, don't walk to the nearest engineer and like have them set that up. Because what that basically does is it means that Google won't index your codes and also the coupon sites like the deal spotters and all of that stuff like retailmenot, whatever, they won't find it. So that's one way to combat it. The whole like how, how big of a piece of the puzzle or pie are they? The. It's different for. I've actually seen a lot of reports. Like sometimes we'll pick stuff up. So we're a consultancy, we're actually not like a traditional agency. Sometimes we'll pick stuff up from agencies and we'll see like, hey, this one show like performed outsized to what we normally expect from it. And then we start digging in and we're like, hey guys, your code is everywhere on the Internet. This show is actually not performing very well. So it's a little bit of art and science, like it's a little bit of detective work. Actually one of the women I work with, we joke around that we use our superpowers that we've gotten from like Googling people on the Internet. Like if you give her some somebody's last name and the place and year they went to high school, like she'll find you. So it's like that kind of like Google searching. Like you're just getting in there and like figuring out like, okay, where is this coupon code leaked? And then proportionately you can see like, okay, how long has it been on there? Right? So you can also like, you can use the way back machine to like look and see how long it's been active on a coupon site or a deal site. Usually you can look at shopify conversions and say, oh well, last Wednesday from two conversions a day, this code went to 20. Something happened. What happened? So there are ways to isolate it. I would say for most advertisers it's a negligible piece of the pie. But again, if you're usually an advertiser that doesn't discount and then you decide to launch a discount in this one channel and you don't do it in a way, like we commonly troubleshoot this with our advertisers where we go through and basically like checklist, are you setting this all up the right way and is it, is it isolated enough to not ruin the experiment and, and then go from there? But it's, there's ways to do it so it's not a large proportion of the pie. And some of it is just being really diligent about monitoring your data and explaining any anomalies in the data. Because sometimes this is the, this is the reason, but sometimes, you know, in the inverse way, you'll see a huge drop off of conversions and it's like, oh, somebody hit something in the vanity URL, got turned off for that period of time and you're like, like, that's why you've really got to be looking at this stuff on a, on a very frequent basis.

Those are some great tactical tips. I want to change the topic a little bit back, like eight or nine years ago in podcast, there was like, a lot of like, hidden gems where you could, you know, sign a contract with a, with a podcaster and like, really get a good deal. Is that still true or is the market for podcast just, like, really efficient now and you're sort of just going to like, you pay roughly the same amount for the quality of impressions? Like, how do you think about the market for podcast advertising today now?

So I would say that there are largely established market rates for some of the top performing properties, but you still have to know what to buy. We, there's millions of podcasts out there. We regularly buy somewhere between 7 and 8,000 top. And those are the podcasts that have both scale and performance, you know, in, in some way. Right. They drive some sort of converting activity or some sort of visit activity, some sort of basically consideration, but more importantly, like, purchases and like, strong intent. It's no, like, listen, 10, you know, eight or nine years ago, we were buying Joe Rogan for like, pennies on the dollar for what you buy it now. And he was still achieving substantial reach and people still followed him and, and were as like, devoted to him as they are now. Right. So it's part of it is like, no, you're not going to ever replicate any, any channel when it's super early and it's monetization. You're never going to replicate that initial, like, shotgun approach to media planning. Right. Just by virtue of being in the channel. Like, you win. But that's why it's important people who know what the hell they're doing. It's also why, like, this drives me nuts. But people are like, oh, you could buy this stuff programmatically now it's like, yes, what you are buying programmatically, you are buying my leftovers.

Yep.

Because the really solid performers that we renew and buy year in and year out, first of all, some of them sell out. It is a fixed amount of inventory that we're Talking about. Right? So if, if a show has four mid rolls and they publish 25 episodes a year, you know, it's roughly bi weekly. They take a week off for Christmas. That's 50 episodes times four, that's 200. That's it. So you know, it's like if you, if those 200 spots are sold direct to advertisers and most advertisers buy on a 17x cadence and I'm bad at math, but that's not a lot of advertisers that can buy a show. And so it really becomes more about working with the people who know what they're doing in this industry in order to buy it. Either hiring them or hiring somebody partial like us or you know, again, like it's just don't go direct to vendor necessarily. There are some instances where that's appropriate. If you have like somebody with either offline experience or like vendor direct buying experience, like partnership experience, to that extent they can kind of muddle their way through podcast for a minute. Like we've actually published a lot on the topic that people can use and like choose their own adventure. But if you want to scale and you want repeatable results there, you still kind of need to know what to buy, which networks to work with, how to structure the buys is more important now because they're, you know, the industry has kind of grown up around this, around this.

There's like a million follow up questions that I have on this. But I want to ask this question because we talk to a lot of brands that are like, okay, we've had a lot of success on the digital side and now we like want to explore offline, but we're not sure how. And really I want to know, like, what do you advise a brand for? Like that initial test that they should run. Right. How do they figure out, like, is podcast a channel worth exploring more for us or not? You know, and how do you go and answer that question as a brand that doesn't have a lot of experience here?

Yeah, I think the, the answ answer is to not onesie twosie it. A lot of the time I'll have a conversation with brands and you know, we'll be trading notes, whatever. And they're like, I did this one sports podcast three years ago and it tanked. So podcast doesn't work for me. And I'm just like, okay, it is now a $2 billion a year industry. I mean depending on who you talk to. I don't think it's actually that big, but that's A whole other. We. That's a whole other soapbox you don't want me to get on. But it's like any other channel. It's like, are you going to test one placement and one like ad content group and meta and be like, moment? It doesn't. For me, no. You need to make sure that it's like a stable enough, large enough experiment that tests enough variables that you're like, okay, I see a feasible path to scale here. Or I don't. Right. Which is a good. It's an okay result. One of the nicest compliments I ever got from a client was when they were like, I've worked with three agencies and I don't know if podcast works for me. Within three months we were like, let me save you money and time in the future. It doesn't work for you. And they were like, thank you. Like, it was, they were like, this is the best thing that any of our partners have done for us because now I can deploy my funds and, and efforts elsewhere. Right? Like now I can go explore direct mail. I can go explore this. Because like, they had a bias towards thinking podcasting was going to work. It was the right consumer, wrong media type.

So what, what metrics were you looking at in that experiment? Was it, was it a post checkout survey or was it a coupon code or like, what were you using? We know that combination.

Survey.

Got it.

Combination coupon code and post checkout survey. But in this instance they didn't have a strong offer. They couldn't do one. So what we also wound up using was like a pixel based implementation. And again, this is where it's like, okay, we were paying attention to all of, all of, all of these mechanisms were telling us a similar story. And then we also matched it up with back end first party data. Right? So it really is just about like data triangulation and knowing like there's latency in this channel when it comes to consumption. So knowing that like you have to. For people who have not done an on demand medium before, done advertising in an on demand medium, it takes a second. Because we're so used to instant gratification from like Meta and Google. Right? You know, I was actually just talking to somebody this week and he's like, yeah, dude, put a grand in on Meta. Next day you got three. It's great. I'm like, well, just to let you know, that's not the typical experience from anybody anymore. You know, there's been some algorithmic changes and some of Apple's privacy changes have resulted in meta's efficiency kind of, you know, not being what it used to be as a platform. I think every marketer who's here and in general know that essentially I want to click.

But you mentioned.

Wait, really quick. Let me answer your question with like 20 seconds of Michael. So 20 seconds of how do you test podcast in a way that feels like you will get the right, you know, stuff from, from the experiment, the right, the right results, whether or not it's positive or negative. We're just looking at viability. We are testing this channel for viability in your marketing mix. So we often pay back on the first run. But I tell advertisers all the time, don't count on it. We are placement testing, we are variable testing. When you're doing multivariate testing, you can't imagine you're going to nail every variable at the outset. What we might find is it's the first time you've ever explained your product in an audio only fashion versus video and audio. So guess what? It's a little different. You're going to have to adapt your messaging, you're going to have to adapt your landing page. You're going to have to adapt stuff. So what I tell people is the minimum starting point in terms of live media. You want to do somewhere between eight to 10 weeks of live media. You want to do at least three integrations or episodes per show. And you don't really. Marci. It is exactly like what you talk to people on tv but like, you're just like, okay, patience. Like we need a little bit of time in order to accumulate enough reach and frequency. Which again, this is where as marketers we have abandoned some of the old concepts of like C, like I just explained Cube. But nobody who is in digital advertising is going to be like, yeah, Kyum, I got it. But when you start explaining it as like, you are accumulating enough impressions so that you are hitting the right reach of the right population and you are talking to those people enough times to make traction happen, all of a sudden it's like, wow, that is an advertising concept that makes sense to me. Thank. It happens in every medium. It's just, it's different for on demand channels because like, you know, there's probably a bunch of podcast episodes dropping during this call. I'm hoping that nobody is like, oh my God, I saw this new episode of Smart List. Let me drop everything I'm doing. I'm not going to pay attention to work. I'm going to consume this right now and I'M going to take the action that one of the brands tells me to take right now. Like that is not how anything works. So when it, when it comes to like duration of flight and everything, somewhere between eight to 10 weeks. Because you want to space stuff out enough, you want to spend somewhere in total, I would say a minimum starting point is somewhere between 75 to 100k. Optimally you're spending usually a little over a hundred in order to test enough vertical or genres. But we've genres and podcasts are meaningless by the way. I don't know who is coming up with those at Apple and how they're classifying them. It's a lot of it's self classified but like what the hell is society and culture, right? Am I right? So what we've actually done is we've re verticalized all of the shows that we buy all thousands of them into like, like, I mean some of it is like borderline insanely unhinged name. Like we have different true crime genres and it's like true crime, spooky, true crime, paranormal, true crime, murder. But all of the consumers in those different subsections of true crime are different and they behave differently. And some of them are more effective for certain brands than others. But what you get for that spend is you're going to work with somewhere between five to 10 publishers, probably 20 to 30 shows and you're probably going to be looking at like somewhere between 50 to 75 drops. So that's. And that is enough. It's enough. What you're going to hear a lot from agencies is that it's a quarter million dollar a quarter minimum, which is bullshit, frankly. It's not. The reason that it is, that is because traditionally in this channel and in many offline channels, agencies are compensated for based on commission, which if you back out to 50 a quarter into a 15% gross media commission, what winds up happening is that's how they can essentially make enough fee to be able to support the program. We, I don't believe in that model. I, we divorced. You know, you only need to spend a hundred thousand dollars in media. You need a senior team like that. We work on retainer, so it's a little different. But it doesn't incentivize us essentially to spend a certain amount of money in order to get the result from the experiment that you want. And that's what you're looking for, right? As a growth marketer, I want to test as little as possible to get results that are meaningful enough that I can use Moving forward and use to scale and diversify my media method.

Christina, I want to get to my last question before we have to wrap here, which is I want you to. So you mentioned Pixelist, which seems very obvious how that works for digital platforms, but how does that practically work in these offline media? And someone asked a question about IP matchback. So how do those work? How well do they work? How trustworthy are they? How do they fit into this picture of doing measurement?

All I wanted for years was for somebody to replicate pixel based measurement in offline media and it has been replicated to a certain extent. So some of it is server side, some of it is Matchback and it's exactly what the, the question in the chat detailed. It can be effective. The challenge that I've found is we've done a lot of head to head tests of a lot of the platforms that are out there and despite having similar methodologies, they all tell us something different. It's driving, it's honestly it's driving me crazy. It is, it is effective for some channels more than others. It's probably, I would say the most effective for podcast and streaming audio. Obviously you know, is, is ID certified and all that stuff. But I would say like streaming audio and podcast, fairly, fairly solid television. You don't need to use pixel based measurement. You do like spike based methodology for the most part is what. Like it's, it's always like some sort of modified spike analysis. We're actually testing some stuff with a radio partner right now to see if we can get at basically an accurate, for want of a better word, pixel based measurement. But it's, there's like multiple ways. So some of this is, you know, you're actually using a one by one pixel. It's embedded et cetera in the, in the feed. Some of is prefix URL. So it's the, the actual tracking mechanism is. Is. Is a pre. Is in the prefix URL. But there's different. I feel like I'm talking so fast because I'm like I have so much to tell you. It's like any or the media buy might suck, I don't know. But like pixel based measurement and server side measurement. It's, it's unfortunately I wish it were. It is not a silver bullet. Maybe it's not yet a silver bullet, I don't know. But when I look at stuff like the, the deprecation of cookies that's coming, the changes that are coming when I, I look and I see everything that's happened with Apple in the last few years, I'm like, guys, I don't know if this is it. You paying attention to the rest of the media, like you paying attention to general advertising over there. You guys want to keep marching down this path? We're going to march down this path. Okay, cool. Like, that's why I'm a big believer in first party data and in using, like in using modeling on first party data in order to understand what's actually happening in your marketing mix when it's possible. I am all about using and triangulating performance with third parties. But ultimately, at the end of the day, first party data is going to be the most resilient in the face of all these privacy changes. So I am unequivocally biased towards it.

I think, I think that's a great place to wrap. I mean, it sounds like, I think probably what we advocate for a lot, which is you have to look at, you should look at all of the different measures. There's probably value in most of them, but you also have to be super skeptical. Right. And you have to sort of look like, take them and bring your smarts and then use that as a way to actually make decisions. And with that, we are out of time. So thank you, Christina. First of all, this was so fun. You're so smart. We could have done this for like three more hours. I'm very, I'm very sure about it. The call to action for everyone else. As you're signing off here, follow myself and Christina on LinkedIn. We're both pretty active there, share a lot of stuff, so hopefully y' all find that interesting. Check out RSU agency@rightsideup.com and then sign up for the next coffee break, which we're going to do with an academic, Dr. Cohen Powells, who's a professor of marketing science. We're going to get super into the weeds. It's going to be a fun one. Thank you all so much. Have a great rest of your Thursday and a great rest of your weekend. Weekend. Thank y'. All. Bye bye. Break. Which we're going to do with an academic, Dr. Cohen Powells, who's a professor of marketing science. We're going to get super into the weeds. It's going to be a fun one. Thank you all so much. Have a great rest of your Thursday and a great rest of your weekend. Weekend. Thank you all. Bye bye.