Attribution Model Patterns with Yorgos Moschovis - AgileData #46

Welcome to the Agile Data Podcast, where we talk about the merging of agile and data ways of working in a simply magical way. Welcome to the Agile Data Podcast. I'm Shane Gibson.

And I'm Giorgos Moskovis.

Hey, Giorgos, great to have you on the show. You have the privilege of being the first partner from the Agile Data Network coming on the podcast. We've been working together, Agile Data and DataCenix, building out this consent management offering. But today we're not here here to talk about that. Today we're here to talk about attribution. I think we're talking about marketing attribution, but actually I got no idea what attribution is. So we'll probably rip in with that question first. But before we do that, why don't you tell the audience a little bit about yourself, your background, how you got into this lovely world of data and analytics?

Yeah, thanks, Shane. It's good to be here. Look, I started in New Zealand and that's how we met at the Office of the Auditor General. And then I discovered that I had an inclination to do a lot more analytics. Joined AC, then Telecom, now Spark New Zealand. Then suddenly I discovered that I could do all of this as a consultant and joined Silicon Graphics as a consultant. That took me to Asia. Once Silicon Graphics got into some real financial problems, luckily I got tapped on the shoulder to join a SaaS person. Who was exiting from SAS and starting an Asia-Pacific version of Unica, the campaign management software, which was later sold to IBM. And past that, I joined one of our major clients in Singapore, where my base became for 12 years, then became a chief of analytics for a major bank there, then moved to Singtel to unify all of their analytics. And that's how I came across the problem of attribution since they had a hell of a lot of channels in introducing digital. After 12 years in Singapore, I was headhunted to join Fairfax Media in Australia, which was a hell of a lot more attribution problems, but in a good way because they are a mechanism to sell the media offerings to various clients. After Fairfax Media merged with Nine Entertainment, I discovered that I wanted to do a lot more of this and joined a small company that did only attribution. As the chief of analytics there and chief of operations later on, I must have done maybe 20 or 30 different projects on attribution in various industries. And that's how I started liking the kind of challenge that attribution came with. And that's where we're at today. The attribution problems have become a little bit, a little bit more confronting since the demise of third-party cookies and a little bit more complex. Since the invasion of walled gardens. It's all good stuff. Nothing stays the same in this space, that's for sure.

And before we get into the attribution, effectively you've gone from being like a doer to a leader to a consultant, and then you've pretty much done most industries that have been around. So you've seen a lot of it. Think about back though, what the hell is it, 30 years ago when you first started out doing the analytics at the Office of the Auditor General. What was the term back there? Was it, were we back in the wave of data mining? Was that, or were we just back in, it's called statistics. I can't remember. What was the term for people that combined analytics and data together using stats back then?

I think the killer term was data mining in those days. Most of those terms are coined by big organizations, software or hardware that wants to market their stuff. Around the planet, right? Data mining was the term. I don't think that statistical modeling was, was sexy enough, because everybody hated statistics in those days.

I remember doing stats at school and going, the same for me. And then I, at one stage I joined SAS, the statistical software company, and went, oh my God, actually I was wrong when I said to my, my teacher, I'm never going to use this crap. She was like, Damn. It's funny how even back in the UNICA days, the dream, the vision from the software industry was self-service. Anybody can model, anybody can do stats, anybody can do analytics. And what are we, 15, 20 years on and we're still trying to sell that dream. Maybe the large language models are a true realization, but who knows? So let's get back to attribution. Let's start out with a definition. What the hell is it?

Yeah, look, in simple terms, it's a mechanism to figure out which marketing channel is probably the better one to achieve a specific financial objective. And usually when I say financial objective, it's because attribution models have to be linked to a specific financial event, uh, some kind of purchase or some kind of non-purchase or whatever. That's the way that all modeling works. It just requires some kind of well-defined response variable. Once you have an event like this, attribution looks at finding out what the contribution of each channel participating might be, and that helps you fund them. So if you feel that TikTok is the channel that has the greatest contribution to purchases of your product or service, then should be funded a little bit differently. From other channels.

So let me play that back to you in the language I tend to use. So what you're saying is there's a core business event. So who does what? Let's just use the e-commerce one and then we can go into other industries. Customer orders product. The customer comes on the web store, they add a product into the shopping cart and they buy it.

So core business process.

And what you're saying is as an organization, that's where we get value from. People buy more stuff. So we want more of that to happen. So we want to understand which channel drove that process, which channel was the thing that influenced that person to buy that product. Did I get that part of it right?

Yes, that's right. So the initial reaction is that the last step was the one that created all the value. That's the Amazon model. You look at a specific ad somewhere and with one click you buy. And therefore, in this simple example, the ad and the e-commerce software will share the spoils in terms of contribution to the purchase event. In real life, somebody maybe came to your website via search, via Facebook, via TikTok, via a variety of other channels, and they should get some kind of credit.

Okay. So before we get onto that complexity around last step, first step, and all those other words I've read lots about, but understand little, we go back to that core idea of attribution. We see people come in and do the core business processes or events that we care about as an organization. And so we want to answer the question, when they did that, where did they come from? You know, what contributed to them achieving that process? So it's kind of like the core business question we're asking. And then quite importantly, what you said was the action we want to take is we want to put more money or we want to focus on those channels to achieve more people following that process. So these channels we want to put money into because we want our outcome is more people buy more things and therefore we get more money. So that's the flow is understand what drove the behavior to, for them to do that thing. That's important to us. And then the action is how can we encourage more people via that channel to do that again? Did I get it right?

Yep. That's right. That's the way it works.

And then we get into the complexity. So that's where we start using specific terms like last step. Before we do that though, does attribution only ever work with marketing? When you talk attribution, are we always talking a marketing use case or Is the term attribution like a pattern that can be used for other industries, other use cases? It's just typically used in marketing.

Yeah, it can be used in, in other situations as well. When you have a series of events that culminate in a particular objective that you are tracking.

And you could probably apply it to product analytics. So if I know that when people come into a software as a service and there's a core feature or core outcome that they need to do from that, then the way they flow through the events they do within the application, I could attribute which ones are driving them to that core feature that I know makes them sticky or not. So this idea is it's just a series of events. Those events are done in certain orders and we want to attribute which event is the thing that's driving the value for us, the process or the action that we're expecting that user or customer to take. And therefore we know one's working and these other ones aren't. So we either pump more money into the ones working or we fix the ones that aren't. That's the core of the process.

Yes, that's right. So analytically, they are the same.

Let's go into that idea of last step. So I can see that pattern in my head, I think. So if I play it back, the last thing we saw them do was the thing that drove them doing the thing. So the last thing they used to get to the shopping cart or the website to put the product in the cart and buy it we attribute the last step was the thing that drove that behavior. So that's the easy model because we know there's only one jump. They did this thing, then they did this thing. We're going to attribute that last step to the thing that drove the value. Is that right?

That's right. Typically in a digital event in particular, the last few steps are digital. They happen on the site. So it's easy to say the last step is my site because that's where it all happened.

All right. So that one's easy. As long as you can figure out how to get all the event data, rack and stack it, figure out what was a customer or a unique person. Figure out what the last one was done, make sure all your time zones and your UCCs are lined up, that you're not dropping any events, that you're tracking all the events, that they can't get to the shopping cart through an event that you're not tracking. All those simple data problems, right, which nobody has. So how much of your time when you were doing those attribution models was dealing with that data mess? All those things I just flippantly said aren't a problem, which we know are. I'm assuming like all data problems, a lot of the time was sorting out the event data, there are sequencing, UDC dates, quality, all those things.

Yeah, there is a fair bit going on there, but most of these problems are now solvable, a lot more than they were just 2 or 3 years ago. Just sequencing events or creating a big data pipeline, if you want to approach it in that sort of detail, are not as hard to solve as they used to be. Nobody should be scared in those circumstances. I think it gets a little bit more terrifying when you have data from different sources, like when you're mixing channels that are online and offline, that can become really complicated because there are different rules and not the same gradient of detail.

So before we jump into that one, let's just focus on online, because that's not easy anymore. In the past, we had little evil cookies that would drop down on my device and I could go in through many channels. And many sites and you'd always be able to see it was me. I couldn't hide from you. And so therefore you could assign events that happened to me knowing it's me. Whereas now with all the privacy stuff coming in, those cookies aren't readily available. Identifying a person and then attributing an event or a channel or a set of channels to them, even in the online world, has become incredibly difficult all of a sudden.

That's true. There is a glimmer of hope. Trying to do this on a per-channel basis, because most of them will provide enough data for you to be able to do that. Not PIA data, I might add, but enough events and enough datetime stamps for you to create something. It is almost hopeless to try to stitch a journey from one channel to the next while identifying a single even device, let alone person. I will say identifying a person is impossible, but even the device is tricky.

One thing I am seeing though is I'm seeing a bigger push by organizations to create third-party data and assign it against the event. And what I mean by that is that when you're coming in through a single online channel and, for example, we know there's some way we can differentiate your event or your session from somebody else's, so maybe you're logging into something that we control and we know about, or maybe there's some other way of identifying it. Organizations are starting to tag these identifiers themselves against those events so that they have this first-party record where they can align these events and attribute them to a person or an individual or a device. That first-party data's become more and more valuable, and therefore there's also more work for them to tag each of the things they're doing, each of the channels with this first-party data so that they have a little bit more data to group by. Is that what you're seeing?

Yeah, not only that, but companies— there is a whole new industry that has emerged on identity management. The job that it's doing is exactly that. They specialize in trying to figure out what is a device that can remain constant across different digital consumer journeys.

When we talk about online channels, give me some examples of what you would call an online channel.

Obviously, an organization's own website is an online channel. But also social media. You can bunch them all up together if you want to have some kind of summing up or identify Facebook, TikTok, Reddit, whatever you want separately. All these can be done, but these are the majority of online channels. I like thinking of email as a kind of offline channel. It's still big in situations where you have almost real-time events. It's almost shameful not to class email an online channel since you get an email as soon as you have behaved in a particular way.

And then when we talk about those social media channels, do we differentiate between an advert and organic view? Do we treat that as a sub-channel? What's the term that people normally use if they're trying to differentiate attribution to ad versus organic?

It's exactly that. Usually this conversation is not necessarily based on technical considerations. So if it starts from, let's say, the CFO or the Chief Marketing Officer going backwards, if this particular person wants to fund organic search separately from ads, then you should differentiate between the two. If internally there is no budgetary line to differentiate between the two, then there isn't the same kind of pressure to do that. Although I will suggest that the insights on consumer behavior will come if you do differentiate whether there is separate funding or not. I err towards more detail rather than less detail.

We all do. I remember the days when we had to buy expensive Teradata boxes and we had to be really careful about what we load. And then the whole big data bullshit wave arrived. But one of the benefits out of big data was the idea of data lakes, and relatively cheap storage where we can store most things and keep ourselves comfortable that it's all sitting there in that messy cupboard if we ever want to try and find it. But I just want to come back to that key theme. So that key theme there is we attribute to a channel when we want to actually be able to invest in that channel. So if we don't care about breaking our investment down between ads on Facebook and organic Facebook, that's not the way we run our operating model. And we don't care about that, then there's some value from an analytics point of view, but from an attribution point of view, we don't care. We're figuring out the boundary of where we want to attribute somebody came from and we want to invest in that attribution, that channel to say, do more of it or less of it. That's always the key outcome that we're looking for.

Yeah, that's right. I've worked with some organizations that were adamant that they weren't interested in separating out a lot of social channels, and that meant a lot more simplicity in their modeling because you didn't have to distinguish all of these social media channels. And this type of attitude changed within 6 months from a situation where there was no desire to fund separately TikTok or Reddit. There was a request to understand what is going on with a view to maybe funding those in the future. It's actually a lot more useful to differentiate them since you can anyway, technically, rather than just bunch them all up together.

So what drove that? In the BI world, we have the standing joke that the first question you ask data for is never your actual question. How many customers do we have? That's actually never the question. It's just to give you a boundary to go, okay, what type of customers are they? Ah, so those types of customers, where do they live and what do they buy? It's the first question you're going to answer to frame the next question. So in that scenario where they went, okay, we only care about these channels at a mega level. And then it's a sudden, okay, now we need to be a bit more fine-grained. What drove that change?

I think the dominant marketing thinking became realization that you had to be wherever your customers were. If you thought your customers and prospects were on TikTok, then you have to be there. And that was the winning argument within organizations to say, hey, Why don't we start by finding out what's happening with that channel and maybe identify how many of our customers or prospects live there? That's what drove it in the end. You just couldn't ignore that there were more and more channels around. But the reality also is that it's so easy to start a channel now. And you go back to your thinking about big data and data lakes. You have a lot of different data points. Are they all going to be useful 6 months from now?

But if we take that example, the value stream I heard was, okay, there's a new channel out there. Let's say that we're not doing anything in the TikTok space. It's just a channel that we haven't touched yet. So we go and say, okay, there's a channel out there. There seems to be a large amount of people on that channel. Everybody's going to TikTok, so we should go there too. Effectively, the first question then is, is our audience that we care about there? And then you do some research, you go, yep, there's some people there that look like the type of people that are our customers. And then you're probably going to go, okay, we need a bit of an experiment. So let's take a small sum of money. Let's experiment with some creative, let's push some content out. But then what happens is we've thrown some money at this channel. We've got no idea whether we got traction. So attribution tells us, attribution says we funded this amount of money on that channel and these customers have been attributed. They've done this action. Yeah, they bought this product and we attribute back that channel is what drove them to do that.

Yeah, I've seen in a couple of our projects, I did see small traffic, yeah, for some of these channels, no doubt about that. But when you track the journeys all the way to the financial objective and the conversion event, it was rather significant. So you'd say, okay, I paid $1,000 to have some creative there. And I looked at the clicks and blah, blah, blah. Why don't I make it $2,000? So that's doubling the budget in a way, proportionally speaking. If your budget is $2 billion, $2,000 is not a big deal, but analytically, you have just doubled the budget for one particular channel.

That takes us on to offline and online. If I think back to my data mining days, is that the same as when we used to call it above the line and below the line? Is that the same term or is that different?

I think you got me here. I didn't think. Of offline, online on those terms, but yeah, you could say that.

We used to talk about when we did ads on television, it was above the line because we had no data. We couldn't attribute anybody actually watching it, let alone anybody buying off it. And then below the line was something that we, like a website click, where we could always see a behavior. We actually had some data that we could, in theory, attribute to a person.

You can use that analogy. And yes, these problems still remain, though a lot more smart TVs are in the market and you can figure out who's watching what anyway. Or a lot more than just a few years ago. But yeah, you can use different techniques. They're not exact techniques, but you can use different techniques just like you would have done in the old days with out-of-home. Say you have an ad that is in a particular location. So you're thinking that everyone who drives past that location will have seen the billboard. You have no idea whether they did or didn't.

Is that the definition, the standard definition of online and offline? So online is when it's a digital device. And it's a website, social app, that kind of thing. And offline is when there is no digital device. Is that the definition between online and offline from an attribution point of view?

No, it can still be a digital type of channel, but you just cannot get hold of the data in real time. And therefore it behaves just like a channel that could be like out-of-home billboard. You get some data from what happened a month ago in a particular digital location. So you can't use this in real time.

So it's about the frequency of that data, the availability of it. Can we make a decision on it right now because the person's doing something right now, or is it something that we'll use to train some models later that might make a recommendation, but it's not a near real-time recommendation for them?

Yeah. In my head, if I can create a big data pipeline with date and timestamps, that would make it an online channel.

Last step. This is the last thing they do. We attribute that channel where they took that step as the thing that got us success. What are some of the other attribution models that are out there and how are they different?

The opposite of that, first step, which is first step, last step, right? Marks for authenticity. Who came up with this genius?

I came. But the question, is there a quick step?

It must be something that you can do on Excel. I know you like Excel spreadsheets, so I thought I'd throw it in.

Ah, I'm modern now. I've got Google Sheets, mate. We're on a Google stack now. Excel, Microsoft's dead. Alrighty. So first step is—

first step is very useful in understanding whether there is a channel at the top of your marketing funnel when you're playing for reach. That is dominant. For example, is it Facebook or TikTok or Reddit or one of the news websites in the country? First step is important in understanding that is almost always an entry point here into the consumer journey that I'm interested in.

And then I'm guessing there's some stuff in the middle though. I guess that people don't just go to one channel, which is their first channel, go to a second channel, which is their last channel, and then buy the product. There's a whole lot of messy behavior in the middle, which is where the complexity comes in.

Yes, that's right. What customers do and what consumers do is quite complex because they don't even move in a particular linear fashion from the top channel to the converting channel. They can be toing and froing for a decent number of days. And in some occasions, like in some industries, think about cruises, your consumer journey might be months there in the making as you try to figure out where to go. What is the right route, whether you have enough friends to go with you. And it can take months to plan a trip like this. And this is reflected in a consumer journey.

Do you end up doing some kind of cohort analysis or segmentation where there is short consumer journeys with a small number of touches, and then these long consumer journeys with a large amount of touches, and therefore you're grouping those into different models to say these types of customers take the long path, these types of customers take the short path, therefore, we we may pump money into the channel that's on the first step in the short path because the time between us spending the money and recovering that money from a product purchase is, is what we care about right now. Or you could look at it and say, okay, in the cruise case, people that do the short journey take a 7-day cruise and, and a cheap boat. Somebody that does that long journey, they're investing a lot of research, they're doing the 28-day cruise and the luxury boat, so therefore we don't get our money back from that channel as quickly, but the margin we're getting off it is much higher. So that's kind of what you're doing is using that event data and that behavioral data to then decide where your bang for buck is.

Yeah, that's right. That's exactly how and why you do it. I am more interested in the shorter consumer journeys because they don't have the same data complexity. Even first-party cookies disappear after a few days now. If you wanted to track very long consumer journeys, you have to have some marketing approaches in there to get people to authenticate themselves, log in, have some goodies along the way, create some interactions to keep the interest going. You just cannot think that you're going to track a 7-month consumer journey across different types of channels.

A little bit of complexity I hadn't thought about. So as well as having a bunch of online channels and no safe way of saying that's the same person engaging with those channels, and then a bunch of offline channels where we may or may not get data at the right time to help us with that attribution. We're also saying that unlike a corporate system where that data pretty much survives forever, that ability to reference that data disappears after a certain point of time, after a certain number of days, we lose the trail, so to speak, of that person, even if we are able to figure out what they're doing. So that urgency brings in some complexity in terms of the way we create these models.

Yes, that's right. And in fact, it gives birth to a different thinking about what model to work with, simply because some work a lot better with continuous touches. Others work a lot better with deeper histories, but not a lot of touches.

Explain a little bit the difference between a continuous touch and a deeper history, because I kind of think there are two patterns sitting in there with those two different terms.

Maybe you think about a consumer who is buying, let's say, health insurance. Typically you'd buy health insurance once a year, maybe twice a year if you are unhappy with your provider. In order to understand the customer layer, you have to look at deeper histories, see what the type of product that they have bought has been this year, last year, 3 years ago, and so on, and see how their circumstances have changed. So it's that few touches, deeper histories. And if you compare this to buying, say, supermarket goods, you would do this twice a week, maybe every day in some of the situations for fresh fruit and veg. So you have shorter consumer journeys and a lot more touches. So you don't need deep histories. You don't need to analyze what you bought at the supermarket 12 months ago. You only need to understand what you bought last Sunday.

So it goes back to that volume. If I'm constantly doing something and I'm doing it quite often, then the window of time for a richness of data is shorter. 'cause I'm doing it more often. Whereas if I do it once a year, I have to go back over multiple periods to get any kind of representative data to infer something. And then deeper touches sounds though, we're bringing in some more of the demographic attributes about people and things versus the behavioral ones around what they're actually doing. Is that right? So we're augmenting the lack of volume of data with some more demographic data that's valuable.

Yes, it's always safe to do that. I have seen in recent times that static demographic data don't play such a major role anymore. I think the reality here is that as more and more people, or nearly everyone, is on a digital channel these days, the static demographics aren't really a differentiator. It's just what happens with your digital behavior that is predictive of the next type of touch. That you may have. But whether you are male or female, or you live in an affluent neighborhood or not, or the locale or age, I don't think it is as crucial as it used to be.

What you're saying is that we're moving away from segmentation models that are based on demographics, how many people in your family, what ethnicity, where you live, what your income is, where you go for a holiday, all those kinds of things that used to infer a propensity to buy. And now we're moving much more towards that consumer journey and that channel and that attribution to say the journey they have to doing this action. So buying that product is this. So therefore that's where we're going to pump the money regardless of the segmentation and demographics. Maybe we'd use the segmentation and demographics for the type of content we're producing and the type of ad or the type of thing we're putting onto those channels to attract them.

And even the communication style. Perhaps, that also matters. And I can stretch this to specific consumer experiences as well, some different service models. But in order to figure out when somebody's going to turn up in the supermarket to buy 5 to 10 different products, I don't think that it matters whether they are ethnically this or that, older or younger or whatever.

So first tip, First time we saw them, top of the channel, last step, last time we saw them before they took that action or the converting channel. What other attribution models are popular?

There are models that are about equal type of credits, as in if you have 4 different channels, each one will get 25% of the credit for a conversion. That's useful in the sense that at least what channels are participating in the process. Others create a gradient. So higher gradient towards the conversion for obvious reasons, or higher gradient towards the first touch again, for obvious reasons. So it's really a situation on what is it that you're going to use your funding for? Are you going to use it to build the upper funnel or are you going to use it in order to boost your revenue line?

But each of those channels are influential. So how the hell do you work out that there's a converting channel, a last step, but actually a certain number of people, if they didn't go through these first steps or they didn't go through these 3 channels, then they never get to conversion because that in my head, that becomes a fairly large matrix model of multiple customer journey paths, multiple touchpoints over multiple periods of time. Over multiple channels in different orders, different times of day. There's just so much variability in that model. Is that where the magic is? Is that where the actual hard part is doing that rather than just go first step, last step, easy, fairly simple business rule. Anything else is an optimization problem, quite a large complex one.

Exactly. And also I find an analogy to what you have just said, because it was, it was actually quite interesting. You think about anonymous journeys. Let's forget about attribution for a moment. If you are browsing anonymously and 3 or 4 days later, you authenticate yourself, then pretty much all software that is available today will stitch the journey back to you and say, okay, now that you've authenticated yourself, I can actually say that you have been here as a repeat visitor 3 or for France. It's the same kind of process. Once you start looking at somebody who has converted and therefore they have had to authenticate themselves at the last step, then you can go backwards and figure out, okay, now I know what it is that you've done before you got here.

But how? The only way you could do that is IP address.

It's IP address, and particularly for your own site, you can, your first party cookie is still allowed.

For a period of time.

For a period of time.

And again, that's getting locked down more and more as Apple and the other people that actually honor privacy, they're locking down from a device point of view to stop people tracking you for longer and longer.

So there is a lot of complexity there, and that's why it's not exact science.

Real world example, I'm just thinking about, I love coffee. I work from home now. I'm not going to go and walk down to my local cap and spend $5 or $6 for a coffee. We're bootstrapping. We can't afford afford that. We're incredibly frugal. So I have an espresso machine. I buy the pods, Espresso New Zealand. I have an account. I get pods delivered. All good. Last year we spent a month in the UK. The UK coffee is far worse than the New Zealand coffee. So I was like, okay, I need an espresso machine. I go onto my Nespresso account. I go, buy me a machine, deliver it to where I'm staying in the UK. It goes, oh, you can't do that. You're in New Zealand. And I'm like, okay, so go in, I create a new account, use a UK address. So I now got two accounts and I go, cool, buy that machine, get it delivered to this address. And it comes back and it goes, oh, there was a problem with your order. So I do the old service help desk call and actually Nespresso, I gotta say they have one of the best responses in terms of the people answering your questions, no matter which channel you contact them. It's awesome. However, they came back and said, oh, it's because you're using a New Zealand credit card. And I'm like, yeah, no, you can't do that. Okay, so we've actually got a UK credit card. So I go back in, I use a UK credit card. It goes, nah, you can't do that. And you look at the help and it goes, oh, it's a debit card. And I go, okay. So you go through and you use a credit card and it goes— eh-eh! Right? No. So in the end, I got somebody in the UK to create an account, log on, buy the bloody machine, get it delivered. That's cool. Going back to the UK again this year and I'm like, All right, I need some pods. Go back on with my UK account. I was like, fuck you. So then I hop on, I go, okay, I turn on my VPN and I go, right, order it. So VPN, I'm in the UK using my UK account with a UK credit card. Nope, still blocks me. So now I'm getting a bit grumpy, so I'm like, okay, So I clear my browser, I go into anonymous mode, I turn my VPN on, I go in and create a new account in the UK with a UK address and a UK credit card, and the order went through. But think about that. How does it know it's me now? They've actually put a whole lot of things in the way, and I'm assuming it's around fraud and all that kind of stuff, right? They don't want a whole bunch of people overseas buying stuff, and I get that. But now they've lost an attribution model because in theory, there's no way of them stitching me back together in my buying behavior.

Yes, that's correct. And I have to admire the anti-money laundering techniques that Nespresso UK is using. You have to admire all this. Essentially, they're saying, yeah, I want you to be somehow traceable to the UK authorities.

As a data person and as a systems person, utmost respect for the systems they have in there. As a consumer, no, they're a bunch of dicks.

That's right. You've made it so difficult. And I'm sure that everyone has a story about this because it's so difficult to behave like yourself if you're going from one country to the other. It's just not happening.

Which is interesting given COVID and global working and everything else we do has become remote, not location-based, which is interesting.

In situations like this, your attribution would be neither available in the UK nor in New Zealand. It would be different silos, which is incredible because that's what attribution sought to do in the first instance, not to use each particular channel as a silo. And as soon as the third-party cookies disappeared, then Everything became a silo or a walled garden in sexier terms.

The good benefit of that is when I went on to one of my connected TV apps that has ads embedded in it because it's free to use, I didn't get a Nespresso ad pop up. So at least their attribution model wasn't working because they weren't spamming me with ads about the thing I already just bought because they didn't know who I was. But that's one of the outcomes of attribution, isn't it? Is that as well as figuring out where to throw money, which channel is the best, you can then actually say, are there any other complementary channels that potentially that person is seeing? So there's some benefit there because they're giving you the message through another medium, another channel that has potentially some influence, but they're also just making it more complex because now I have yet another channel with yet more events to go into the model to figure out which ones drove me to the action they wanted and therefore which one they should invest more in.

This happens, yes. And I think that the promise, if you were in a position to understand the consumer journey and its length and how many touches it had, then you could also figure out what the messaging would be to accelerate somebody who was at the beginning of the journey or the middle. And that would be a tremendous insight for any organization to say, I don't need to put a special in front of you for a cruise because you have just started your journey. You're not in a position to buy now anyway. But what I can do is I can build a story that is going to stick in your head for months. And when the time comes, then I will be number one choice for you.

Let's just talk about the value stream to do this. What I can see is you grab a bunch of events, data from a bunch of channels, you put those events together in one place, you try and infer the concept of a unique ID or identifier for something you wanna bind to those events, typically a person, but maybe a device. Then you just run ChatGPT or a black box neural net against it, and it tells you the answer, right? That has been a little bit flippant, but that's the kind of high-level process, isn't it? Is grab the events, put them together, bind it to a concept. Run some models to figure out how we attribute which event or which channel is driving that behavior, and then make an investment decision on that. That's the core steps in the process.

Yes. And it opens up a lot of other different avenues for operational improvement. Like I said, the messaging can be consistent with where you're at in, along, along the value chain, or it can be an alert as in it's taking you too long. Usually the customers that convert do so in 8 days, and now you are on the 31st day of fiddling with various options and you haven't converted. You optimize your contact process as a byproduct of all this.

And then if we look at that value stream, where are the problems? So we've talked about a couple already. So we've talked about identifying the core concept, the core entity, the person or the device that we want to bind all these events together to get a journey for, that's incredibly difficult. We talked about the fact that there are multiple channels that have multiple events and multiple touches and can be done in any order. And therefore that is an area of complexity. Are there any other areas of complexity that make attribution difficult?

What is changing all the time is what data you can get out of the big social media giants where Even when you put your ads on, quite rightly, they don't want to identify the person who has clicked on your ad and ended up on your website. But the type of detail that they release varies between different organizations as well. So that makes the scientific process a little bit more complicated to figure out. I have a lot of detail from one channel, not a lot of detail from another one. How do I marry the two?

And stop it overweighting. If we have a large number of events that are quite high quality coming from one channel, and then we have a small number of events that are relatively summarized coming from another channel, then when we're modeling, depending on the techniques we use, it's going to skew the model to where there's that richness of data.

Yeah, that's right. You have to make them comparable. And that's just understanding what it is that you're going to get out of each one of your marketing partners. If you get 100,000 events that have the 31st of March as a date, and then you have another data pipeline that has exactly the same volume, but it's daily, you have to do something about that.

That's where the magic happens.

Where the magic happens.

And also explaining it. So I always remember back in the days where neural nets were the bee's knees, uh, data miners or data scientists would often run a bunch of models against some data. So maybe we were doing, I dunno, prediction or a next best offer or churn predictions, and they would get the data, they'd clean it up. It's always 80% of their time. They'd run a bunch of models. I always have the standard joke that 90% of analytics is a GROUP BY, the next 5% is a linear regression, and then we sometimes get into K-means or something else. When I watched experts they were going in, they were running a bunch of different models. They were then comparing the models, maybe looking at a lift chart or some other way of saying what model's the most effective. They were tweaking the variables on a couple of models to see if they could train it on that data better. And then they'd come out with this magic answer that says, okay, this model, bang for buck, is the best model to run against that data to answer these questions. And nobody could understand it. One of the techniques they do that I really liked was they then typically ran a decision tree and they would say to stakeholder, this was not the model we are running, but this model is gonna give you a representative idea of the factors that are influencing the model. Maybe it's a churn model and then the decision tree's saying, it's people who have rung the call center 5 times in the last month who have had a bill change where their prices have gone up and defaulted on one payment. That's the tree. They're the ones that are highly likely to churn. We're not actually using that model, but it's a model that can visualize that flow for us. I'm imagining that with attribution models, we've got the same problem. There's so many variables, so many demographic behavior variables, so many time-bound variables, so many different ways of modeling it that actually explaining how we're attributing is going to be a fricking nightmare to anybody that didn't create the model, or even the people who did create the model. Is that true?

It's true, but the market has moved on from that. There isn't that much emphasis anymore to explain what this particular statistic does or how the model operates. And the reason for this, in my view, is because it has become so easy to do A/B testing. And it is actually a much more efficient process. So for example, if you say that this group of customers is your best probability of conversion, you can just run with this for a few days. To see if your control group versus the target group have any notable differences. That's your A/B test. Within a few days or even a month of putting it in market, you'll know whether the insight was correct or not. That's the same with attribution. You just say that, okay, I'll probably increase the funding for a particular channel by 5%, and then I'll see how I went. After a few days or a few weeks of operation, and then I will know whether I got it right or not. It's a lot more pointless today to get in a room with 10 people and flood the screen with statistics and say, oh, this statistic is 0.34, which is a lot less than 0.9. Why is it a lot less? It doesn't sound like a lot less. And therefore X, Y, or Z is going to happen. I haven't had it within the last 4 or 5 years.

That means we're getting agile approaches to that kind of work. Like we no longer should sit in a room for 9 months building an enterprise data model that nobody can execute. We need other ways of doing things quickly, iterating it, showing the value, and then testing it.

Yeah. And it is a much easier engagement process with all the stakeholders. Say, hey, look, I tell you what, I've done the best that I could here. But the real test is when we put some specific offers in front of the consumer, see how it goes.

But that also means that model has to move from being a research model to a production model. Let's take your example. We're going to go and say that there's 4 channels. We're going to use a gradient model where there's a percentage assigned to each channel. That's how we inferred the most valuable channel where we should do the next investment to achieve that outcome, which is people buy more products. We're gonna go do some testing. We're gonna throw a bit of money in that channel for a while and see where we get uplift in the number of people buying our product. That means that model actually has to go and be flipped into a model that we can use to measure whether we've seen an increase in people buying products from that channel. That kind of is really important now, cuz we have to measure that investment achieve the goal that we thought it might.

It's been the promise all along. The promise all along has been You build a model for a particular operational improvement, not to satisfy yourself that you've built a great model.

Sometimes in the past, models have been built to justify the investment that's already been made.

That's why I'm laughing, because I've been there too. Build this particular model, $3 million later, it's fantastic.

But I tell you what, we're not going to implement it because something else has happened and I don't really want I always wondered when we saw the analytical maturity model where we always talked about descriptive and predictive and all that kind of stuff, I was always amazed that there wasn't something on the axis at -0 that was post-rationalization models. We'd already done the action and achieved nothing, but now we wanted to create a model to prove that it was the right thing that was done.

Yeah, I must say the, this type of meticulous work, and let me be clear, this is how I learned my craft. As well. I haven't turned my back to that, but it's just not appropriate for marketing and consumer type of events. It's more appropriate for risk pharmaceuticals, um, where getting it right before you have actually implemented it is 10,000 times more important. But here we're talking about what is your downside risk. Your downside risk is that you can potentially annoy people by putting in front of them an ad that is not appropriate. Okay. It's true. But the reality here is that you can also stop doing that a month later after you have figured out whether it works or not.

I can see there's a spectrum of rigor that you need depending on what action you're going to take and the outcome from that action.

If you're talking about— everyone is talking about actionable insights. Well, here it is. You can action them by actually going to market with a specific approach and then come back a month later and say, okay, something didn't work. I'm going to have to look at how I derived my understanding of the consumer journey and conversion events, or change the way I have made assumptions about funding by 5% or 10% increments.

All right. So just to close it out, I often still think about the maturity model for analytics, the idea of descriptive, predictive. In the past, I've always said optimization is one of the most complex models to deal with. Simple example that's horrible to try and model is you have a school, it's an in-person school. You have a bunch of classrooms, you have a bunch of teachers, you have a bunch of subjects, and you have a bunch of students. And now you're trying to optimize that the classroom's always full. With a certain number of students and the right teacher for the right subject. And actually when you try and do that algorithm, it's a nightmare because there's so many moving parts. It's an optimization problem. So attribution to me sounds like an optimization problem in terms of complexity of variables coming in, the complexity of demographic versus behavior, the events, all that kind of stuff. It is quite a complex area to actually build models on. Is that true? It is one of the harder ones to do?

Yes. And that's one of the reasons why I wanted to talk about the RISA rulebook, because it's a great learning experience. It keeps you sharp. It is because of the fact that it has a lot of moving parts, not despite it. And the stakes are high. Even if you get it right for 6 months out of 12 in a year, think about it like this, for some organization, in the US in particular, their marketing budget is $50 million. Just getting the same number of sales with 10% less is a big deal.

I'm a great fan of agile, as you would expect, and small bits of work, value early, feedback, iterate, and build on it as you go. So, I'm assuming that if you wanted to start from somewhere, starting with a last step model, a last step attribution model would be a good starting point. Build that one first, understand what that looks like, make some small investments based on that, and then move into some of the more complex attribution models over time when there's value in that and you're going to return it based on more targeted spend.

I understand your point. Your point is how can I be practical about this? Because I cannot or will not dive into this straight away. So I will say that first and last touch are probably the two specific data points that are going to give you a lot more understanding of your consumers. But if you do only one or the other, you're not going to be better off than where you are today. Even if it is just a line between first touch and last touch, at least you have a line. With just one point, you don't have a line. I will never advocate building a model based on just one touch. It's just too random. Start with two and see how you go. Maybe the entire consumer journey is three. Maybe your two is good enough, but don't just do one.

Out of interest, how often is it just a three-step or three-touch journey? Common or is it rare?

I was surprised to find out that for one of the organizations that I looked at, it was two or three touches within a week. And people would buy articles between $5 and $50. So for this type of retail consumer goods, I was quite surprised that it concluded so quickly. Not maybe not so surprised about the number of touches, but how quickly it concluded.

So again, it's going to be dependent on the industry, the product, the business model, the customer demographics, the market. The country, there's going to be a whole bunch of variables that influence your customer journey, how many touches there are, and therefore the complexity of the attribution model.

Yeah, an organization can figure out the touches, even if it is within its own site anyway. So it gives you an idea of whether even within your own site, it's a complex journey anyway. So if it is complex within your own website, right? There is, there is a learning there to start with. Yeah. But think about yourself as a consumer. How many touches have you needed for your car insurance?

It's one of the ones I don't change very often.

There you go.

Happy with the company. And so that one, not so much. My internet provider annoys the hell out of me. That one's constant touching. I'm a great fan of charcoal barbecuing, long barbecues. And so there's a remote meat delivery company in New Zealand, Woodies, call out to Woodies. And every time they email me with a newsletter with some meat, that touch, my customer journey becomes short. Oh yeah, I could try that with that. And then I'm on and I buy it.

So there you go. Is it $5 to $15?

They've got this good trick that is free delivery over a certain amount. So you always want to get above that because somehow you think you get value because you're saving $15 while you add $30 to the shopping cart. So I think my total cart value probably is normally $50 to $80. I can't remember what the limit is, but they definitely have some good marketing and sales techniques to increase the value of the cart.

Also, there you go. You can do $50 to $80 within 2 or 3 touches within 1 week.

Probably 1 touch.

Yeah. It's consistent with what I saw. The journey is always influenced by the buying price as well. So if you have, if you're buying a car, which is expensive, then you want a few more touches. That's why the initial approach would be figure out how many touches are required when you can count those touches. Let's say that's your GROUP BY statement right there.

Although it's interesting because with that example, my first touch was actually somebody I follow on Twitter who is better at long cooks than I am, and he constantly publishes photos of those long cooks and recipes. And he was the one that mentioned them. For them to attribute it, the first touch, they have to know that I follow this person on Twitter, that person mentioned them, and as a result of that, I then hit their website and did my first trial order. So again, if you think about the complexity of the events with that, half that data they probably don't have. So again, As we know, often data is the biggest constraint to anything we want to do in the data and analytics world. And in that case, they're probably never going to guess what the first touch for me was.

That's a fair point. But also it may be that by just funding Twitter a little bit more, they can get a lot more of you and that's all they need.

That's a good point. They can't anymore because it's now X and it's locked down and there's no APIs and nobody's actually on there. It's a ghost town. So yeah. Probably attribution from X and taking some of your budget and pushing it out through that channel, probably not happening so much. If your attribution model is still heavily weighted towards the thing formerly called Twitter, you probably need to get contact your boss and get them in to retrain that model for you. But anyway, on that note, if people wanted to get ahold of you, where do you live? Talking analytics, how do people get ahold of you?

I ride there quite a bit. I started liking doing that, or jorvos@datasennex.com.au. Excellent.

Thanks for taking us through that. Attribution is definitely one we've talked about a few times, one that I never quite understood. It seemed like another one of those dark arts. For me, I like this idea that you just want to figure out which channels are driving the business process you want. So if you're an e-commerce company, it's customers buying your products. And then why are we doing that? We're doing that because we want to figure out where to pump our money. So which channel is the best channel to put more money in, which will help us achieve that process happening more often and therefore more business value. And then after that, everything else is a data and analytics problem. And in terms of this one, it's a pretty horrible one. Yeah, maybe get some help from people who have done it before or take a while. That was a great session. I hope everybody has a simply magical day.

Likewise, take care.

That, data magicians, was another Agile Data podcast. If you'd like to learn more on applying an agile way of working to your data and analytics, head over to agiledata.io.