Andrew Tindall — When Not to Test Creativity, AI Prediction, and the Real Signal of Effectiveness

Boringly Effective, the podcast.

Welcome to Boringly Effective, a podcast exploring the marketing fundamentals that actually move businesses. I'm Andrei Tarnovski, and today I'm speaking with Andrew Tyndall from System One. We explore what the evidence says about effective creativity, how emotional responses shape advertising performance, and where AI might help or hinder the creative process. If you're curious about how better advertising gets made and measured, this conversation is for you. Andrei?

Yes.

Thank you very much for coming to the Romanian Creativity Pavilion. Honored to have you here and congrats for your recent upgrade in your career.

Thank you, mate.

What's the question you don't get asked enough about when it comes to your own category, your own profession? I mean, about testing, about marketing effectiveness. Sure.

Good question. to measure creativity. Everyone always wants to talk to me about measuring creativity, the power of creativity, and the value of creative data. And now as our industry develops and gets better, we are putting creative data into econometrics, programmatic media buying, all this stuff. But people never ask me when to not test creativity, 'cause I work at a creative testing business. That's a strange question to ask me, but I've got an answer that I'd like to share for it.

Okay.

We create a lot of shit as an industry. A lot of assets these days, especially with agentic AI.

Yeah.

We need to feed social platforms more and more content. I would say only about the top 10% of that is going to get enough reach, enough investment, enough consistency to even do anything. I would not waste creative data on this 90% wash of shit. I would only bother focusing on that 10%, if I'm completely honest, because most of the tools you get that bottom 90% aren't really there yet. There's a lot of people selling AI creative testing stuff that's just like an LLM plugin that's made up and will change anytime you bang it in. So I don't think the industry is quite there yet in testing that wash of stuff. But System One might be bringing out a tool that might help them with that, which I've been working on eventually.

Can you tell us more about this tool or is it top secret yet?

Well, look, this, our industry is, the creative measurement industry has been changed by AI, obviously.

Yeah.

And predicting creative effectiveness via models versus using human responses. So far, I think a lot of those AI testing tools have been, as I said, either trained on shite data or simply LLMs, you know, OpenAI Claude plugins that are guessing. And from their vast, you know, model. System 1 have been testing every TV ad with millions of people for 10, 20 years now. So we've got this humongous dataset of real emotional responses to hundreds of thousands of assets. And we've trained a model to basically predict that, which we'll be launching in a few weeks.

So it would basically help you in predicting what could work, but could never replace real testing?

Completely. To really understand whether something's gonna carry the creative weight, part of the response to that is surprise and how it's different to what's come in the past. AI struggles at that stuff. So to really get to the exact answer now, the technology will change. You know, AI is the worst it's ever gonna be now, obviously.

Mm-hmm.

But right now, from what I've seen whilst building this model and these products with our brilliant team, it does a good enough job to give you a little bit of a read. Once you work like this. But no, you still need real human system 1 data. So behavior, emotional data to really understand how ads are performing.

So it's that small difference that is always surprising, that is unpredictable.

Exactly. The most interesting stuff is the difference between the AI and the human response. I have a hypothesis that that is actually going to really explain how good an ad is. That subtle difference. But yeah.

Since you're testing other brands' ads, how would you rate from 1 to—

1 to 5 is our test. 1 to 5.

How would you rate your own LinkedIn content since you have millions of reach?

Yeah, no, good question. I'm sorry, but it's a 5.9. And I have the data to prove it. This is AI creator testing platform called Favicon, which like pulls in all the mass engagement and reach data and follower count and how much you're growing, but also how much genuine engagement you've got with the audience. Someone sent me the link to that the other day, and I am 100 out of 100. I have a perfect score.

This is impressive.

No one else does. So I'm technically top of the pile, which I like telling people because B2B creators need all the support they can get because I meet famous Instagram B2C influencers and creators who have like 10,000 followers and it's their day job. They're getting paid so much by brands. You're telling me that I literally reach a million decision senior business people every week. And when they make a decision, it's to take their whole fucking tech stack, which is worth, you know, a billion quid. Or I'm meeting someone whose, you know, whole career is based on 10,000 consumer followers. Bullshit to that. People should be really leaning into the power of B2B creators and fucking paying us more.

Speaking of B2B, is there anything that you at System One have Maybe in development or maybe an existing tool useful specifically for B2B people?

We do B2B creative testing as well where we find decision makers or— what was a fun one I worked on before? We have to find the audience really to test the ad with. Or you build these psychographic matching audiences because it's very expensive to pay CFOs to come do some fucking research. So there's an art to it, but it's possible. I worked on one at the moment where we had to recruit someone that buys chemicals exclusively for fucking dentists or something like that.

Super niche.

Yeah. So it's, yeah, it's just about finding the audience. But I would encourage anyone who does work in that space to not care so much about the audience and the sample. What I actually find is that good advertising makes a broad set of people, anyone, feel something and it interests them. If you can make your B2B content, B2B ad, B2B campaign, your brand interesting to someone that doesn't buy chemicals at your dentist, then you've got something genuinely category growing and will work on everyone. So I actually wouldn't bother, you know, because often you have to spend, you know, £10,000 to recruit 100 CFOs or some shite. And, you know, an alternative solution to that would be, well, don't worry, guys, we've built the synthetic audience of CFOs from an LLM. Yeah, right, you have. You know, I wouldn't worry about that. I'd focus on the fundamentals of advertising and comms strategy, which is, you know, making your advertising interesting, emotional, and distinctive to everyone, because that will do great things. It allows you to, when you do broad media buying, It will be interesting to everyone. But if you put that into engagement algorithms, algorithm reach for organic is based on who's interacting, who's engaging. So if you've got content that's broadly interesting, all those people are interacting with it, it's gonna push your content further in an algorithm as well. So yeah.

Any piece of content needs to get your emotions moving.

Yeah.

Am I right in saying that it doesn't matter if it's exclusively positive? It can be both, or maybe even asymmetrical in favor of not so pleasant emotions? Or does there need to be a balance?

It's a good question. I get this a lot. Fundamentally, emotional engagement is the ground floor of effectiveness, i.e., not feeling nothing. Apathy, neutrality, dull. You can't feel nothing. So sure, if it's that I'll piss someone off, piss someone off. However, the affect heuristic. So we have these heuristics that basically allow us to go about our lives, that govern our behavior. They're learned behaviors that we make in split seconds. One of them is an affect heuristic. The affect heuristic is if something comes to mind and it feels good, we do it. Similar to the familiarity heuristic, which is if something's familiar, it feels safe and we do it. So sadly, that means that advertising that leaves people feeling more positive is more effective. I've seen this in TikTok data, in Effie data, in IPA data, in fucking customer sales data, in the amount of car brands that enter the street every week depending on how positive the ads were. Like, it's anytime I look at it, sadly, and you know, creatives hate this because, oh, I'm gonna have to make a fucking happy clappy app.

Yeah.

And even, you know, there's, there's a whole broad range of positive emotion. There's pride, there's schadenfreude, there's humor, there's pleasure for others, there's sensory pleasure. There's all sorts of different positive things. And what's important as well is the expert creatives that understand this lean into jeopardy. So surprise and negativity, but give a positive ending. And that makes the ending even more positive. So look, emotion's important. There's a huge cost of dull. But sadly, because of the way our human brains have been wired for the past millions of years, positivity goes a lot generally. There'll always be fucking exceptions, you know, depending on the strategy, category, and brand. But generally, positivity does much better, especially when building on Some, as you mentioned. Jeopardy.

Perfect. You recently said that the— let me see if I can quote you correctly— the all-you-can-eat AI buffet has closed.

It is true. I was very fucking depressed about AI for like 6 months.

Wow.

It feels like it's a big tech race and it's going to change everything. Which is true. That is still true. It is going to change everything. It's changed marketing and advertising. It's changed research. It's changed coding. It's changed marketing science, machine learning, engineering. It's changed all industries. And so it should. It's a step forward in technology and we should lean into that. Why I'm not depressed about it anymore is it's clearly going to end up in a commodity. There's enough players, thankfully, And global governance should be pushing for more competition in this space, to be honest. There's enough players where everyone's going to have fucking AI and their own LLM. I do not believe this will not end. Well, this has to end in a price war. It's going to be, okay, which one are you using? Fucking, I'm on Claude this week. Why? They're doing $9.99 for 3 months, so I swapped back. That's what it has to do. I don't fucking believe that these players are going to different— they can't differentiate themselves. So they're trying. And then where that ends up in, yeah, price war, but also a marketing war. And you're seeing the ripples of that now where Claude is leaning into human, more human AI, really conversational and is a friend. Whereas ChatGPT is going kind of mass, their almost way with it, and futuristic. So yeah, it'll end up in a price war and brand is the only way to get out of that. And but you're seeing the fact that people are realizing this because, you know, we've burnt through all our fucking credits. I use Claude to build a lot of my research because it's fucking amazing when you plug it into Excel. Like, Jesus, the shit I'm able to do now is insane. And it's made me a better researcher. But I'm on the phone to my IT team every week like, look, mate, I need some more fucking credits. Can you just load up the account? I'm trying to change the world here. Open it up. Every Sunday evening in my pants. Come on, mate. Fucking credits. So we burn through this stuff way quicker than we think. So yeah, we all now need to start thinking about profitable use of AI. Not can AI do this, but should AI do this?

And speaking about should AI do this, lately more and more luxury brands started doing all human work. TV commercials filmed By hand, so to speak, animations done like they used to do them in the '60s and so on. Do you believe this could reach the mass market, or will this remain only in the domain of luxury as a way to be distinctive?

We have seen this. This isn't new. Aardman. Do you know who Aardman are? Chicken Run? Wallace and Gromit? Do you know these guys?

Okay.

Aardman is an animation studio that does most of the animation with clay figures.

Okay.

And you can see the thumbprints in the clay.

Yeah.

When they do it. Barbour. Do you know Barbour? The luxury brand? They've done all their Christmas ads by hiring Aardman to do it with clay because it signals craft, effort, costly signaling. This took time. You can see the fingerprints in the fucking doll, you know? That's marketing strategy. That's comms strategy. AI is just another fucking camera. It's just one way of making ads. Some people will use it. Some people will go, oh, I think that one of the ways we can signal that we're premium is this way. Like, you know, it's nothing new. So yeah, some people will use real, some people use AI, some people use a blend.

And now a quickfire round. Stars in System 1 or shares on any social media?

Oh yeah, engagement rate is bollocks. So stars, obviously.

Emotion or attention?

Emotion.

Because it drives attention?

And attention. Yes, because it orientates attention and because attention doesn't explain alone brand memories and the industry needs to move to brand memories. That's what we care about building. You know, I can have a 1-second bit of attention or a 5-second bit of attention. The 5-second bit of attention might not be branded. It might not make me feel something. That 1-second bit of attention might be heavily branded and make me feel something. I'd want that 1 second of attention.

A tricky one. Byron Sharp or Orlando Wood?

Orlando Wood.

Because?

Because Orlando was not an asshole.

We can cut this if you want to.

No, I don't care.

And the last one. Gin or rum?

Oh God, I didn't— I'm not flummoxed on any of them until this one. Rum. I've really enjoyed running some of the world's biggest rum brands in my career.

This is why I'm asking.

I've not worked on— I worked on a very small niche gin once, and oh God, gin's boring. Rum has character, story, age, flavor.

Thank you so much for this.

Thank you.

That was Andrew Tyndall from System 1 on Boringly Effective. If the conversation gave you a new way to think about creativity, effectiveness, or AI in advertising, subscribe wherever you, you listen to your podcasts. Boringly Effective is a Summable production. I'm Andrei Tarnovski. Thanks for listening, and I'll see you in the next episode.

Boringly Effective, the podcast.