AI Sucks at Paid Ads. Do This Instead

All right, so we're going to kick this episode off by talking about how and why machine learning isn't what you might think it might be in terms of predicting ad campaign success. So let me explain what that means first, and then we're going to go through a paper here, and then Neil and I, we're going to jump through a couple different ideas that we have for today's topics. But basically, let's say you're Facebook or Meta, okay? Let's say you're Meta. Let's say you're Google. You have a huge corpus of data.

Yeah.

And you see a lot of people running billions of ads out there, right? So you have a sense for what tends to work and what doesn't. Now, you would assume that these machine learning algorithms would have— would do a good job of predicting the next campaign success, right? And like, I would assume the same thing too, because you work off of data. But here's an interesting thing. This paper, and I read part of this paper, I won't say I read the whole thing here, but Neil, can you see my screen?

Yeah.

Okay.

Yes.

So here's what it says over here. For those of you that haven't subscribed on YouTube, go subscribe on YouTube because we're just going to get better and better there. But this tweet says, fascinating paper. It finds that even with access to rich datasets of user-level features, sophisticated ad engagement propensity scoring techniques fail to approximate the causal lift results of RCTs. RCTs, just call it a test, right? So RCT just means like a test. That's all it means at the end of the day. Just think of it as an experiment. And in some cases, dramatically overestimate them. So this graph over here that those of you that are watching right now, this just shows like in the RCTs in red. So that's the original test, right? That's the control version basically. And it shows these are the other like machine learning models. And this is like another model over here. But the wider the range is, the larger the discrepancy. So you can see That the more these— the discrepancy starts to get pretty big as more and more, let's call it the lift deciles. I'm not a scientist, but these are the projected lift estimates. So it could be anywhere from, look, 3% here to 20%. And if you have such a wide range, you're basically not predicting anything at the end of the day. That's what this thing is calling out here. I'm trying to really simplify it for people. And this is predicting it for upper funnel, mid-funnel, and lower funnel. And you can read this entire paper. I'm not going to read the whole thing here, but if you— I'm going to move to sharing this tab instead. You can download this PDF. Can you see this, Neil?

Yeah, I can see it.

Okay. So I'm not going to read it, but I'm going to tell you who actually helped with this paper. So it was done by the Kellogg School of Management at Northwestern University. It was also done by someone on the Meta Ads research team. And then this was also done by somebody else at the Kellogg School of Management team. And so basically the conclusion is, hey, even with all this sophisticated tooling and sophisticated machine learning models, at the end of the day, creativity cannot be predicted. That is the takeaway here.

The other big takeaway that I would have to say is from all the years I've run ads, and I've been doing this for 23-ish years now, I've learned one thing. That has been consistently true. And what most people do with advertising is they look to see what worked for the competitors or worked in the past and they keep redoing the same thing over and over again. Similar to the concept of banner blindness, we typically see the best performing ads are ones that are unique and are doing something new and fresh that others haven't really tried. Or if they tried it, it hasn't gone mainstream yet where everyone's just doing it. So these new type of ads, these new type of strategies, they're the ones that tend to do well. And it's hard to use machine learning data to predict that. And I'll give you a prime example of this. So back in the day, if you were selling a product like a bar of soap, you would typically just put it in the grocery store or run ads on Google saying, check out this bar of soap, doesn't leave your skin dry, you know, has lotion in there, has antibacterial. stuff so you don't have to worry about getting sick as much, yada, yada, yada. Then the way they started selling bars of soaps for new companies was, hey, let's try to do those Squatty Potty type of ads. So you guys may have seen like the Lume ads or the Squatty Potty ads where they'll create a funny type of commercial that is very humorous. Like with Squatty Potty, the unicorn pooed sprinkles.

Rainbow ice cream.

Rainbow ice cream, there you go. And that converted extremely well and they generated millions and millions of dollars in revenue. How do you use machine learning and AI? And I'm not saying it's impossible, I'm just, you know, I'm not that much of an expert in machine learning. I'm not actually any expert in machine learning or AI. So I don't know how you actually input data and get it to predict what's going to work in the future when what works in the future is usually going to be something drastically different from everything that we've seen. Continually over the last 23 years in marketing.

So here's the interesting thing. So, and then we can move on to the next topic, whatever you have queued up next, Neil. But so it says, here's the abstract over here. So at the top of the paper it says, and by the way, the title of this, not this experiment, but I guess you could call this an experiment, is Close Enough? A Large Scale Exploration of Non-Experimental Approaches to Advertising Management. So you can just go Google that. But here's what it says. So with access to over 5,000 user-level features, these data are richer than what most advertisers or their measurement partners can access. Overall, despite having access to large-scale experiments and rich user-level data, we are unable to reliably estimate an ad campaign's causal effect. So same thing to what Neil's saying, same thing with what I've seen with our clients and our experimenting in the past. Sure, you can try to draw inspiration from other people. We always recommend Creating a swipe file, looking at what other people are doing, but don't try to copy it word for word. Don't try to do the same thing. You have to add your own spin to it and that's how it's going to be successful at the end of the day. That's where creativity comes from. Creativity can build on what other people are doing, but if you try to copy it exactly, you're going to end up just chasing other people at the end of the day and that is by default not being creative. Yeah.

And speaking of advertising, dude, is it me or do you feel like the whole world is turning into one big ad?

I feel like yes, the whole, you know, you explain more what that means, but I get what you're saying. Dude, okay.

So I was looking at The Rock's profile, all right? And this is on Instagram that I was specifically looking at his profile. And when I was looking at The Rock's profile, I was like, okay, you know, I was like, what portion of his posts are actually promoting a company he's involved with or brand he's involved with or movie he's doing or anything like that, right? And 10 of the— Last 12 posts, no joke, were promoting a company. In essence, it's pretty much an ad. Now, I'm not saying he's getting paid for it because some of these companies he owns, but in essence, he's still promoting something. And it's not just The Rock. If you look at what you and I do on social media, we may put out mainly educational content that's not promoting anything, but we're hoping that it helps us build a following, which it has over the years, nowhere near the size of The Rock. And a portion of those people will be like, wow, people like Neil and Eric put out decent stuff. They're smart. What company do they work for? Oh, they have their own marketing agency. Oh, can I hire Neil or can I hire Eric? Right? And it's created business for you and I. And even though we're not directly saying, hey, sign up for our ad agency, we're still indirectly promoting it. And it's not just with that. It's just like everything. You go to airport now and this has been like this for years now, but you know, it's becoming more and more popular in most terminals because I fly so much internationally. You see HSBC on the ramp from the airport to the airplane because you walk on that bridge thing and you see HSBC ads everywhere. Like everything is literally turning into some sort of ad and there's people are promoting stuff. And you now even have AI influencers like Lil Miquela, who's like made money from Prada, BMW, I think Gucci as well. The Uber app, you know, itself has ads in it. If you take an Uber, some of them have those old school taxi cab, like triangle boxes on the top of their roof and they're putting ads on there as well. Or in the backseat of the Uber, you see a little screen with ads. Like everything is turning into advertisement. And I don't necessarily think it's a bad thing. I just think it's reality. But what's funny is, is I was talking to someone the other day, they're like, man, this is getting overwhelming, it's too much. I'm like, dude, instead of worrying about it or complaining about it, because you can't really do much, just jump on the bandwagon. It's either are you going to make more sales by doing what everyone else is doing, or are you just going to skip out on the sales and the revenue?

You can't do much by complaining. You can only do much by doing. So look, at the end of the day, Neil, do you know who Peter Drucker is?

Yes.

Remember that name? Peter Drucker. So, I have a book over here, by the way. Look, it's Managing for Results. I have this over here. This is sitting right next to me. But anyway, Peter Drucker, he's like a godfather, old school management guy, right? And he's wrote some really good books. How to Manage Oneself is a really good one. It's only 50 pages. Anybody listening to that should read it. It's pretty simple. But my point of this is Peter Drucker said there's only 2 main things that matter with business. Neil, do you know what they are?

No.

Okay. So one is marketing.

Huh? I was just saying, I'm assuming you're going to tell me.

Yeah, I'm going to tell you. Here it is. So one is marketing, 2 is innovation. Those are the 2 main things that matter in a business. And this guy's done like a lot of stuff, right? Written a lot of books and all that. And so what we're saying at the end of the day is everything is marketing. And like we've talked about how we believe there's going to be a rise in creator operators. And Neil and myself, we're very much creator operators. So we operate our business, like that's 90% of our time. But we spend maybe 10, 20% of our time creating content. There's going to be a lot more of that because marketing, yes, marketing is getting harder, but still everything is marketing at the end of the day. So what do people need to do? People need to adapt or they're going to die as a business. So that is that. Are you good on that one before we move on?

Yeah, let's move on.

Okay. So next one. So Neil, have you read about this one? So there have been some big media layoffs in 2023. So when you think about Vice, when you think about BuzzFeed, when you think about like there's just, A lot, a lot of media layoffs, right? And I think it's been like the amount of layoffs increased by like 78% or something like that year on year. And we want to talk about what that means for marketing in general. So I can start it off first and then we can go back and forth on this. But here's the thing, right? It's one, media companies, they're like the old media companies, especially the ones that haven't adapted. They're getting crushed because the new way of doing media, it might be like on TikTok or Instagram or on other mediums, YouTube for example, right? Or maybe people are just starting their own Substacks, whatever it is exactly. It's just, it's changing, right? The other thing is with the advent of AI, the cost per writer or the cost for writing in general, it's getting a lot cheaper, right? And so people are really sticking around with the best people that they have and maybe they're letting go of the people that are maybe B and C players. Because they only want to focus on the top tier. So those are just 2 things that are leading to this. What else you got, Neil?

Dude, with BuzzFeed, have you seen their stock? It went from $10 when they came out, somewhere around $10, $9 and change or $10, whatever you want to call it, to now $0.25 as of today.

Oh my God.

You know, and to give you an idea—

Oh my God, dude, down 97% in 5 years.

$35 million market cap. It used to be where they had a $1.4 billion market cap. This is like, it's ridiculous on how much it dropped, but the reality has kicked in. One, a lot of these media companies are really relying on one or two channels. Two, one of the algorithms, you know, well, this is actually one still. When some of the algorithms change, they get crushed. And 2, they were growing by spending tons of money and they weren't really profitable. And at the end of the day, people were like, you can have all this revenue, but if you can't make profit, who the heck cares? It's just a lot of overvalued companies and they've taken beatings and continually will take beatings over time.

Dude, check this out. So look, media companies have slashed over 20,000 jobs in 2023, okay? And then here's another one I'm reading from Poynter. I guess this is Dave Poynter, maybe, maybe not. But 2023 was the worst year for news business since the pandemic. pandemic. It's a sharp reversal from the last 2 years when layoffs fell and some news outlets even expanded. So CNN laid off hundreds of employees. Gannett cut its division by 6%. Washington Post, NBCUniversal, ABC News, all announced layoffs. And look, dude, hold on. Let me just share my screen. This is crazy. Here, hold on, hold on, hold on. So do you see this?

Yes.

Okay. Check this out. So Adweek cut 10% or so. Alabama Media Group more than 100. Barstool Sports nearly 25%, so 100 people or so. CBS-owned stations, 17, probably more than that. And then 16% at CoinDesk, CNBC, Condé Nast, about 270 people. I think this number is going to continue to rise. And this just goes to reinforce for everyone that if you don't adapt your marketing, especially in this rapidly changing market, you're going to be left behind, right? And nobody wants to have layoffs happen at the end of the day because it's really painful for That your people, it's also really painful for your business.

The funny thing with CNBC is it— when the economy gets bad, people still watch the stock market channels like CNBC and Bloomberg, you know, a lot, just because they want to know what's happening when it's both good and bad. So they're still getting tons of viewership.

Yeah, but look, this is just a lot.

Gannett has been crushed over the years. That company has been struggling for a long time. They haven't figured out how to Gannett. Yeah, whatever you want to call it. I know they bought WordStream, but they've really struggled to figure out what they're going to end up doing over the years.

Yeah. Look, I mean, we're not trying to be doomers here. We're just saying, hey, we're calling things out as they happen and we're trying to help you get ahead of it. And that's all at the end of the day. So look, if you enjoyed this episode, please don't forget to rate, review, subscribe. Subscribe to us on YouTube as well. It helps us a lot. We're planning to push a lot harder into YouTube. Yeah, that is it. See you tomorrow. See you tomorrow.