UNIFY 2019 - Seeing cross-platform in 20/20

I'm Eric Koplikowski. I work at Stitch Fix as the Growth and Media Strategy Manager. Background on my role itself is I work really cross-functionally across our entire media buying teams, which ranges from SEO to offline to social. And really what we do is try to connect and be the glue between our engineering teams and our data science teams. Hi, I'm Dmitry Stavitsky. And background for what Citrix does, if you don't know, we're an online personal styling company that serves clients in the US. We just launched in the UK as of 3 weeks ago. And we serve women, men's, and kids, as well as sub-business lines underneath that, which may be plus maternity or petite as well, and for men big and tall.

Hi everyone. I'm the second person on the panel with a last name that's hard to pronounce. So, Dmitry Stavitsky. I work at Airbnb. I've been there for 6 and a half years, all this time on the growth team. Right now I lead marketing technology, or martech, and what martech does is that together with our engineering, product, data science partners, we're trying to build a platform for growth for the entire company. So there is a product side of growth and there is a marketing side of growth, and I'm on the marketing side, so I'm building tools and systems for marketing to actually run campaigns, measure, attribute, report, automate, do all these things.

Good morning, everyone. I'm Todd Cain, VP of Growth at BarkBox. For those who don't know, BarkBox is a monthly subscription product that launched about 7 years ago and has touched around 2 million customers during those 7 years and designed all the toys and treats in-house to kind of create a special experience between you and your dog every month when you open up the box. Joined the company back October of last year, previously was at Foot Locker. And my team is obsessed with finding new qualified dog parents to bring into the pet subscription product that we have there. Our team works kind of cross-functionally like others here throughout the product teams, customer service teams, merchandising teams to find out and listen to what our customers want. Reply back with that, with a product offering and messaging that really can help serve them a value statement to why they're subscribing to our box. We're so obsessed with listening to what our customers say. Most of the product lines that we've launched, most of the directions that we've gone has been purely off of what our customers have replied back to us in chats and NPS surveys and customer service calls to really support them throughout their journey of, if it's not pet subscription or dog subscription, it's inside of Target or other areas of distribution.

Great. So let's start with channel complexity and different platforms when we talk about cross-platform. So Eric, let's talk about— help us understand what your guys' channel mix is, where your focus is, what's core to your business. Let's start there.

So we were founded in 2011 and primarily grew through organic means and referrals. Back in 2011, mobile app was just a very small twinkle, I think, in Apple's eye. So, we primarily were founded on web. Majority of our media spend, majority of our product iterations have been spent on web. But as we mentioned on the kickoff here, is that it's really important to allow customers to decide on what journey they want to take. So, we're doing a lot more in terms of understanding what types of customers and users should we send to each experience. over time. So again, primarily web, a growing mobile app presence. In terms of channel mix, something I mentioned earlier, we have about 16 to 18 folks on the team, again, contributing to SEO. Offline, that is TV, linear TV, OTT, radio, terrestrial radio, as well as streaming. Then we have the traditional channels, digital channels from Google, from search, primarily on the website, as well as social. So we kind of COVID our bases with trying to engage customers in various different mediums and do various types of segmentation within those platforms as well. All

right, great. Todd, how about BarkBox?

Yeah, similar. Launch as an e-commerce subscription product and purely digital-based. So, with that, the marketing mix historically has been, I think what everyone would probably comment back, a mixture between paid social, affiliate, and Google Search. And I think as the business has expanded, So it's kind of the saturation in those places. So the argument or the saying I always say is that if you're a woman who's 25 to 34 and has a dog and hasn't heard of BarkBox yet, chances are tomorrow you probably will from a Facebook ad. So it's coming. But as we start to look at the business and kind of the experiences that people have when they're absorbing our product, the bottom of the funnel is super important to kind of convert those customers, but the medium and top of funnel, initially, especially in early stages, are sometimes relatively ignored because of the costs or attribution or measurement of those big plays of OTT or streaming or anything along those lines. But as we start to evolve more and more of storytelling of what the business is, it's in its 7th year, so it's not always the newest kid on the block. We have to remind folks and create that experience with them. And I think a lot of that that we're levering in is those non-traditional digital channels that can actually lead to a better experience and ultimately better conversion down the funnel.

So, I think I heard you say we're all in this room gonna get Facebook ads tomorrow morning or even today probably, right?

You might not, but— I

wasn't in the demo.

Yeah, you might not be in there,

yeah. Todd, I'm a dedicated dog dad. I'm expecting to get something moving forward here.

Nice.

Dmitry, you have an interesting perspective on the complexity and dimensional view that you look at. Share with— love to hear that from your perspective.

Yeah, we are equal parts web and apps, which makes it pretty complex. And then at the same time, we are a 2-sided marketplace, which makes it even more complex. But I think channel mix is not the biggest problem about the cross-platform, and I think Cross-platform by itself is not a new thing, but I think the definition of what the platform is has changed over time. And like, to me, I'm Russian and, you know, I have this like Rubik's Cube in my hands almost since I was born. And then I look at this as kind of a multidimensional Rubik's Cube with so many dimensions that come together. And if you think about a couple of businesses under the same umbrella, like Airbnb is In the way that Airbnb is growing right now, every business is a platform. So that makes it cross-platform. And then as a 2-sided marketplace, supply and demand sides of the marketplace, these are also 2 platforms. So probably you need to see across this as well. And then multiple channels, that makes it multiple platform as well. Another level of complexity on top of it. And then product side of growth with all this referral, notifications, incentives channel, etc. That's kind of another layer of complexity on top of that. And then growth marketing side of the growth team with all this search and social and other acquisition channels. One more, here you go. And then there is a user in the middle of all that, and then we want it to be a great experience for the user. At the same time, as a growth team, as a marketing team, we want to properly track and measure and attribute everything. And the question is that how you solve for that, and I guess that's exactly what we are going to get in this panel.

Yeah, absolutely. That sounds daunting. So let's talk about data accuracy as kind of the next phase. Eric, where do you start from a data accuracy standpoint and your perspective? From a marketing perspective, where do you guys start?

Absolutely. I think when I first joined any company, I'd ask, how can I access user-level data? various traits that are tied to a user, be it on the mobile app side, what carrier they're using, how they monetize and engage with platforms over time. So, I think really the core of marketing strategy starts in data strategy and data collection and accuracy of that data. So, a lot of it is just kind of working through your various eng teams. Your analytics engineering teams might be part of your data science team as well. As an organization at Stitch Fix, we're really self-serve for the marketing side. So, it's really up to me and my teammate, Hine, who's here, to really help empower our teams across our entire suite of marketing, not just on growth, to make sure that we're all aligned on the use cases for the data we're bringing on board. Again, just going back to the data strategy informs marketing strategy.

Yeah, no, that's great. Todd, what's your perspective? Where do you guys start? What's your North Star kind of key metrics?

Yeah. I mean, from the subscription side, the key metrics our teams on the acquisition side are super obsessed with are new subscriptions we're generating every day and our CAC, our cost per acquisition. Yeah. And for that, those being our north stars and how the business is tracking that day allows us as much volume and dichotomy we need to spend as much as we need to spend as long as we're in that barometer. And obviously, as we go above the CAC goals of the business, it starts to become more problemsome when you look at the full P&L side of the house. Establishing that within the channels is equally as important to directionally understand how Facebook CAC is tracking that day or affiliate CAC is tracking that day and what the click-through and view-through windows representative of each would be. Within that, it gives us our wiggle room of how we operate every day and plan and operate the business. Think critically about what leverage we can add to that mix to help increase conversion, which will then decrease CAC, or how do we increase spend but know that it's going to counter the kind of CAC goals that we have for that day, but that's okay. So, it's always a kind of mixture and playing between both those houses.

Yeah, that's great. In our last conversation, a couple of you talked about offline. So, let's talk a little bit about offline channels. How do you track that? How do you incorporate that into your metrics? Eric, why don't you start?

Absolutely. I think for us, when I joined Stitch Fix, this was the, I guess, the 3rd company I've worked at that supported offline channels from the mobile app days, the gaming company, to Wish doing a little bit of acquisition through TV. So, essentially what we really had to do is understand with our data science partners, We have a really great acquisition algos team that really helps us drive experimentation. So, as we invest more into, let's say, linear TV, for instance, I'll talk about that. We need to ensure that, one, we're investing in the right network mix, tier mix. So, a lot of times, as you think about cost per acquisition, that's really backed into a CPM at the end of the day. If you're buying network TV through ABC, you're getting $90 CPMs. You're getting really expensive media. And generally for DR work, you can't really spend that much time spending on those channels. So, we had to actually look through different tooling to help support the measurement side. So, we're looking at incremental lift, setting a baseline of traffic. We're actually optimizing towards fixed conversions. It's very deep down funnel. So, it does change a lot of our approaches there. So, it's a mix of tooling, working with different, what we call ACR vendors, which are automated content recognition that will literally read the ad that's playing on screen for you and then track that downstream to an actual result on your website, which is very interesting. And it's a challenging place to convince your data science team that that panel size is actually going to meet the national skew. So, it's a little bit of that. And I think our North Star, to Todd's point, is incrementality. So, everything we do is based off of Randomized control trials, right? Creates challenges with TV because you have to build a synthetic control. So there's a lot of, I guess, art to the way we think about offline. Now for radio, you can do match market much easier. Everyone does it a little bit differently. They can use external data, use your internal data to do so. So I think it's more about how do you align as a team and an org to actually support some of these experimentation frameworks.

Yeah, no, it's very interesting. Todd, what about you?

Yeah, I mean, it's an interesting question. I mean, same as Stitch Fix, TV is really looked at in terms of incrementality. Part of that is aligning with the right partner. If you're gonna actually buy the media yourself or you're looking for a partner who can also buy the media and provide metrics and data, there's a slew of companies that can do it really good and there's dozens of companies that could do it poorly. It really just depends on what the business need of it is. We're also experimenting with other areas of just eyeballs and where those eyeballs are. We're launching physical gift cards this holiday period, which will be very interesting for us to see what the experience is when someone walks into a Walgreens or Duane Reade and actually want to buy a BarkBox for one of their friends and what the gifting moment is, as well as just the general interaction around gifting and referral and the experience of the one-to-one exchange that really can't be Represented perfectly in the digital ecosystem and how we enable our customers who are ultimately our best salespeople to have those same conversations they're probably having every single day, but arm them with a little better materials or better opportunities that make them valued to do so.

Yeah. You both brought up incrementality. So it's a hot topic right now, very difficult topic. Everyone's focusing. Dmitry, how does Airbnb look at incrementality?

I think we were one of the first companies who started to talk about this maybe 5 years ago. And, you know, 5 years ago when you were talking to an ad network about incrementality, they probably didn't understand what you really mean and what you want from them. Like, of course it's 100% incremental. Like, of course it's all brand new users that you would not find anywhere else. Just work with us. And right now it's kind of a thing that everybody's talking about, which is great. And I think we were collectively very successful in pushing Facebook and Google and basically every ad network that you talk right now. Of course, they will try to offer you tools to understand if it's incremental or not, what's the incremental impact, etc. So I think that's kind of a great achievement that we were able to do collectively. But at the same time, we started to experiment with it 5 years ago and we started to unpack the concept of incrementality 5 years ago and this kind of when you run your first series of lift tests, like, it's kind of like it goes up and to the right, like, oh, and now I understand how the world works. And then as you kind of continue running this test, the more you know, the less you understand. And it kind of— like, it went up very fast, but then it goes down even faster. And then, wow, it's actually way more complex than I thought. And then I keep running this lift test and I understand that it's even more complex. And then I understand actually Way less how it works now. So I guess right now there is an opportunity to simplify all that. And I guess the end goal of incrementality conversation is to really understand how the real world looks like. And the way you understand how the real world looks like and how it works and basically like I do X and it causes Y is to basically gather some evidence and then— You collect evidence by experimentation. The way you do experimentation is different for everybody. And speaking about offline and attribution measurement and incremental contribution of all that, I think we were very proud because we are kind of early days growth team and we were very proud that our online side of things is very much evidence-based, but at the same time, the offline side of things was very kind of faith-based. And then faith-based marketing, like if you believe that it works, it works. And then as long as people believe that it works, it kind of works. But over time, it was less and less believers in, you know, offline. And that's why we're trying to move this side of things in the evidence-based space as well, in terms of how you run campaigns, how you measure campaigns, how you experiment with them, how you really understand that X causes Y, the incremental contribution of it. But for offline, it's super early days. There's MMM vendors, MTA vendors, a bunch of different companies that offer you the software that help you run experiments, and then there is a bunch of things that you can do in-house to establish this causal effect, but yeah, it's a lot of things to do.

Yeah, so along those lines, Eric, how do you guys look at channel exhaustion? I mean, at what point have you just exhausted the strategy or the approach to the channel?

Yes, certainly. To Dmitry's point is that incrementality is just a measurement at any given point in time. It changes based off of seasonality. It changes based off of user traits and behaviors. So, I think a consistent testing framework is very, very important. We hold ourselves accountable to running incrementality tests every month for every channel that we can. More challenging when you're running batch markets because you only could run those things and not kind of influence other people's conversion flow and behaviors. So, I think it's kind of ensuring that there's cleanliness in your experimentation framework, communication across your broader teams to understand the true experimental impact, and hopefully the lift you see. Now, not everything goes to plan. There's a big channel that we worked with that's— I can't name it on stage— That was kind of trying to provide analysis in removing outliers. from the actual data analysis. So, when you start playing around with the experimentation, what you set up pre-test, and you change the analysis on the backside, you can kind of create that answer, like Dmitry said, based on whatever you want. So, they're trying to prove value to us, but we actually did the analysis ourselves and found that the actual incremental CPA was 3 times higher. Interesting. So, there's a lot of kind of like truth you need to have in a lot of, I would say, standardization of when you're approaching these kind of tasks.

Yeah, for sure.

I mentioned that we were very successful in pushing Facebook and Google and everybody else in the world to giving us this lift test tools to actually understand incremental impact. Great, good for us, shouldn't settle. Next thing we should be asking for is to what Eric said. Perpetual holdouts. Like, I want 5% lift test holdout always on. Keep telling it to Facebook, keep telling it to Google, keep telling it to ad network. Nobody knows, maybe in 5 years they will roll this out.

Yeah, you need that user-level holdout. That's what it comes down to really, is just ensuring that there's a constant holdout over time. And it does create interesting challenges for the big players as well as even the small guys to actually get in and actually support broader growth marketing orgs that are focused on Driving true incremental impact.

Yeah, I mean, it's a challenge for you as an organization too because you got to figure out what's the business impact of that perpetual holdout, right? Absolutely. That has implications. What about new channels? I mean, how do you go about determining what new channels you want to test?

Yeah, I mean, it really comes down to can we support the level to do incremental holdouts, honestly. I think if I'd go back in time and we didn't have this acquisition algos team, we would probably be at a really big detriment to running our campaigns efficiently. We have a really large podcast program, one of the largest actually direct buyers in the States. We run everywhere. I'm sure you guys have probably heard a few of them for both our women's and men's business. But if we were to run it now, we know that we can't do any type of holdouts in podcasts. So, I think it does hinder us in slight ways when we're trying to really focus on driving that business value that we can directly measured from that lift. So I think it's kind of a little bit of a balance there. And I think, do you have any type of unique inventory? And I think at the end of the day, as a web platform, you're actually more limited than a lot of folks in the space. So mobile app is still slightly fragmented. Most of the time, you know who to avoid, for the most part. So I think it's a continuous process to understand the inventory, the quality of users, as well as

the scale you can get. Gotcha. Dmitry, what about Airbnb from a new channel perspective? How do you guys approach it?

I mean, first of all, we try to stay focused and not spread ourselves too thin. And then, you know, it's the growth goals that you have and the amount of budget or people or time you have, you know, available. When we test something, yeah, like the default question right now is that, can we run a lift test with you guys? And if we can, like, we try to understand like what's exactly your incrementality lift measurement framework. And then we have a couple of data scientists on the team who really understand the space well, and they can, you know, ask ad network like really hard questions, like really dig deep into how you measure that, how you run the test, how you set this up, how many clients did you run lift tests with, what did you see. Like very often they will not tell you exact numbers, but at least you can, you know, gauge And then also the interplay, like how this channel pays with other channels with other tactics and kind of like try to estimate what's the opportunity there. And then yeah, you ask them about scale, you ask them about like what's your proposed framework to actually understand if the channel works or not. Usually it involves budget, timeframe, how many people you need, what's the kind of heavy lifting, who does the heavy lifting, etc. But that's kind of the general approach that we're trying to take.

Great. That's awesome. So let's talk MTA, another big buzzword along with incrementality and cross-platform. Eric, where do you start?

Yeah. I mean, I have certain opinions that have been adopted through our data science team who are very skeptical of MTA. I think if you're really going to make it work— and let me step back and say I'm a big proponent of MTA in general. I think it's really great. The idea of connecting various user traits across platform, across any part of the digital ecosystem as well as the physical world as well. But I think it really comes down to, can you resolve that to a user ID? I think our data science team, and I would agree with them to an extent, that it's hard to trust some of these black box materials. Now, some of them can provide you log level information. You can dig through that stuff, but I think It's still kind of the holy grail, I guess, of marketing to understand that and to really understand your entire touchpoint across the journeys with clean and accurate data in real time. But I think it's still, in my opinion, it still weighs out. And I would have to do a lot of convincing of our algorithms team to actually approach that and solve that. That said, from the user acquisition side, it's hard. If you actually already have acquired users, it's much easier to do cross-channel. Impact in that kind of framework because you can already build your own robots.

Yeah, I mean, I guess the ultimate question is, is it a buzzword or is it real, right? I mean, it's a—

It's a buzzword.

So I would love your perspective, Dmitry. Clearly you just said—

That was more nuanced than mine, I think.

Yeah, you gave the political answer.

Yeah,

we have our head of growth data science here in the audience. I think he can talk about MTA for hours. So if you want an introduction, let me know. I will introduce you. And, that's why I will stay a little high level and kind of take a step back what MTA or any type of attribution is trying to achieve. It's kind of like some teams look at MTA as a way to re-slice the pie, right? And then you kind of have a pie, you have one way that you used to slice it, and then let's just re-slice it in some other way. So, you move from last click to MTA or to something different. But then, it kind of makes one channel look better than another and one team look better to another, but the pie doesn't get bigger. And so if your goal as a growth team is to actually make the pie bigger, you should look at MTA as a tool to help with that. So we are not reslicing the pie, we're actually finding a way to make something new work and kind of making the pie grow bigger. So that's one thing. The second thing about MTA probably is kind of getting back to just this entire concept of incrementality and then understanding not only how incremental that particular channel is, but like what's the interplay between channels. So like if I send push notification first and only start showing retargeting ads after that, what's the incrementality of these 2 channels? If I do it the other way around, like does it change incrementality or not? If I also have, you know, email that I can send and I have a referral program that I can plug and incentive program that I can plug and all these different types of ads that I can run. So basically the ideal thing that we will try to do is basically to understand how the real world looks like and basically what's the interplay of these channels in the real world and basically what X that you can do causes which Y and actually establish this causal effect. MTA is again one way to solve that and as long as it's evidence-based, you run continuous experimentation, understand the interplay, understand the incrementality of Every channel in isolation and then in a combination or a sequence of channels. That's kind of what MTA will hopefully do. Right now, we're still on the kind of an elevated, sophisticated version of last click. Everybody thinks MTA is the future, but again, it's not about reslicing the pie, it's actually helping the company and the business grow.

And just to kind of add on to that as well is that MTA, you would hope that all your channels are incremental that you're running, right? So, I think you If you're going to go with the MTA solution, you need to ensure that you have incremental channels, right? So, you need to run your— you have your channel-level ICPAs, essentially, at the end of the day. And again, those continue to change and continue to test. So, if you're gonna layer in MTA, make sure you're still running incrementality if that's the route you're gonna go in.

Yeah, they have to coexist. Absolutely. Complement each other. It's not one or the other.

Absolutely.

Todd, what's your perspective?

Yeah, I mean, very much share a similar I

agree. Thought, as these guys do as well, around the idea just that it has to add to the business and not be nice to have. It's a phrase and topic that's been around for a long time, and it's almost becoming more and more rudimentary how people are looking at MTA in terms of post-purchase surveys that they're supplementing in kind of very 1990s-esque kind of services to solve a problem which on the other side has lots of sophistication underneath the hood supposedly. On the side of Bark, very similarly, we kind of look at last click by the channel and use that to really help directionally of how we're driving the business. And every month kind of do some attribution loss analysis to understand what the miss was from the over-attributed channels to where we were. And usually it's not too far off. far off. As we experiment more with bigger things like TV and AvHome and things like that, it could potentially help solve some questions to it all, but the solve might not actually solve the sense of adding incremental net new customers and subscribers to our business.

And I think third thing is kind of a tech side of MTA. And then, like you mentioned, it's kind of a buzzword, maybe, maybe not. And then The thing is that I'm sure like a couple years from now there will be another buzzword on the radar and then, you know, MTA is a shiny thing right now and everybody's trying to do that, but a couple of years from now there will be another shiny thing that everybody will be obsessed about. And the thing is, in hindsight, I would do one thing. I would basically structure my data and build the systems in a way that they make it easy to plug any attribution model in. So there is last click right now. You get more sophisticated in the way you do last click and there is MTA today and there is some other MTA V2 tomorrow. But make sure that the fundamental system that powers that and the data in a way it's collected and structured, like it just, like you can put anything on top of it. And then you have everything, like all touches collected across all platforms, everything is timestamped, new shiny toys come comes on the market, let's put it on top and then see what it spits out. Just make this thing very easy and just, you know, plug in and plug out.

Yeah, I love that perspective. All right, well, since we are the most optimized panel and efficient, we have some extra time. So we'd love to, you know, open it up for questions. Anyone in the audience have questions for the panel? Come on.

I've got friends out there who've asked questions.

Oh, Gadi's got a question.

Dan. Oh, Gadi or Dan Barnes?

Yikes.

A lot of the things you guys are talking about are efficient at super high scale. You could do incrementality tests, you can do 5% holdouts, you can do these things if you have high scale. How do you deal with— how would you recommend people here deal with things where they're at lower scale and they still want to understand what the incrementality or the marginal cost or any of the other things that we need to know to be effective. How do you do that when you have lower scale?

That's a good one.

I think the recommendation for low— I mean, it's not a bad thing, right? You should start with something, you start small, hopefully it grows bigger. I think the general advice for lower scale, smaller budget is just not to spread yourselves too thin. Usually it's try one thing, Double down on it, see if it works. If it works, great, try to scale it up. If it doesn't work, shelve it, move on. And then, you know, like, if you— like, you don't want to have 10 underpowered leaf tests, right? Like, you would rather have one leaf test with sufficient power that will give you understanding of, like, what's the real incremental impact. And then if you spread yourself too thin, you will get 10 garbage tests that, you know, will tell you nothing.

I can give an example of a smaller budget that we're running right now. Actually, at Stitch Fix, we're running a— I run our direct mail program, which is strange still, for lack of other work I do, is that we're running something for kids. We're using different modeling from our direct mail partner. But I worked with our data science team and said, hey, we have X amount of budget. Let's call it $100,000. Should I be running a multivariate test here? It's like, no, actually, come on. Let's just power this test fully. Even though we don't know the response rate, we don't know much of what's gonna happen downstream, but I think even at that level of budget, $100,000, which is pretty much nothing in the grand scheme of things, at least for a lot of companies, I should say, it's kind of really defining what that result should be over time. And we don't know if it's gonna actually prove out to be, one, incremental, or two, enough power, but I think it's kind of, if you can have some logical directional results from previous stuff, you can probably get some level of conversion rate and analysis You get to actual power to get stat sig.

Yeah.

You can also get in a very productive argument with your data science team. Like, you don't necessarily need 95%. Like, you know, sometimes 80% is good enough. You know, if it gives you at least some understanding, it's good enough. Like, if it's better than, you know, now, like now you know nothing, but then you will be 80% confident that it doesn't work or if it works, it may be good enough. And again, this conversation may not be very productive with the data science team, but it's something that you can be pushing for.

So you heard it from Airbnb. Challenge your data scientists. It's

99%.

Any other questions? Gadi's got one.

Hey guys. Hello.

Okay.

So I have a question. Basically, it feels like there's 2 opposing forces right now in the industry, or in general. One of them is sophistication of your stack. Dmitry talked about storing as many touchpoints as you can, kind of get ready to more advanced analysis, enable MTA, enable cross-platform. But the other opposing force is privacy, which basically means store nothing, do nothing, know nothing. So how do you guys see that balance between them? And, you know, I'm thinking some things like MTA, sometimes they're just impossible because you can't even see impressions.

Do you want to talk about GDPR? I think GDPR was the best thing that happened to me in the last 6 months. And this was one of the projects that we were working with, working on. And I think it kind of helps to look at these things from the GDPR perspective because, you know, it's kind of probably like the highest standard of user data and all the kind of privacy aspects related to this data. And I think that companies that were lucky enough to be working in Europe and kind of were forced to work on GDPR, I think GDPR sets a really high standard. And the first thing that it does is basically it teaches advertisers and it teaches businesses to actually treat customer data with respect. And then in the way you collect it and the way you process it and then particularly in the way you share it with somebody else. So you start with just kind of respecting like whatever rights they have around their data and I think GDPR again sets very high standard for that. So companies that were forced to work on it, they kind of set themselves for success in the long term because CCPA will also become a very big thing. So If you are an Airbnb user, you can be 100% confident that we treat your data with respect. And then if you want to opt out, if you want to, you know, if you want to allow us to use your data for ad targeting, we're gonna respect that. If you're gonna withdraw this consent or opt out from certain data usage or certain ways we can share your data with, I don't know, Facebook, Google, all these guys, we will respect that. And then— So you're not gonna track me? We will make sure that you're not seeing any ads, your data is not getting passed anywhere. If you want to delete it, we will make sure we delete it. But that's kind of how you start working with these things. And I think this is kind of becoming a standard. So everybody should keep it in mind and then get ready for not GDPR but CCPA and whatever else comes next.

All right. I think we have time for one more question.

I can speak louder.

There you go.

For many of us, we're probably kind of in between this place. By the way, I'm Kayla. I'm from Gatwick. I'm a brand called Mill City. And I'm in a legacy brand where last click dominates still. And I think many of us probably are. So are you having that battle between last click and multi-touch attribution? What are you thinking about those 2? What do you see the future of that being?

Great question.

Hey, so I think most people on the stage here are probably in the room share the similar question that you have, where we've all kind of grown up with this idea of last click kind of being king. And similar to you, I came from a very legacy retailer where Probably the evolution of that will be much longer than the evolution for some more startup-y type companies here. But to kind of answer your question, I think it resonates more with the action of what it will cause than the thought of why we're doing it. So, if the thought of using an MTA model or more sophisticated kind of version than last click is to provide better insights of just knowledge, important then. And obviously to understand how a display ad can affect someone eventually going directly to your site or clicking through a Facebook ad is exciting to learn and probably prove a lot of people right or wrong over the last 5, 10, 15 years. But the action of what it will do after that is probably equally as important and selling that action. even harder to decide that you do want to overinvest in some very top-of-funnel display that will affect conversion later down based on this methodology that not many people understand is a very difficult solve and a very difficult thing to sell. Probably even more difficult when you're a publicly traded company. So all those things kind of have to be considered, but I think that's part of kind of the ongoing conversation with kind of more advanced versions of tracking how customers are buying, of understanding ultimately what does that solve for you as a marketer, and then what levers and valves does that allow you to play with that you ultimately couldn't play with before that can help solve what you're trying to solve.

On Todd's point too, I think it depends on your mix. We're heavily focused on growth at Stitch Fix where brand is becoming a larger part. So, I think the brand team essentially wants to know what kind of impact am I having on path to conversion at the end of the day. So, there are those kind of shifts in your business depending on your media. So, if you're half and half, I think it totally makes sense to invest more time in looking at MPA solutions because you do want to understand, did this content do something? Did getting them to the website do something and help actually tie that to downstream? Because at the end of the day, we do know that consumers don't see brand versus performance. They see your brand speaking to them across all the touchpoints. So, I think it depends on the business, of course. It's something that we're going through as well.

And I think it's not last click versus MTA versus something, and it's not a battle really. And I think it's very unfair to say that, you know, last click is worse than multitouch and multitouch is better than everything else. I think it's kind of coming back again to the question, like, it's like, it's how about— it's about how your world looks like. So, it's very likely that for certain advertisers, last click will be exactly how their world looks like. And basically, that's the question, which of the attribution models is kind of better representation of your world as an advertiser. So, can be multi-touch, can be last click, can be something else, but then as you kind of get to really understanding what my particular world looks like as an advertiser and then which attribution model is the best representation of this world, I think that's the right thing to do and that's kind of Making it not the question of, like, who is better, who is worse, but actually what is the best way to represent your particular world as an advertiser.

Yeah, and I think also what you guys have shared is, even in that regard, there's multiple strategies, right? It's, you know, you have last click, you have multi-touch, you got incrementality. Tomorrow is gonna be something else, right? And it's really pushing the envelope on making sure that you're testing all those things and really building the platform or an organization to support it. So with that, I want to thank the panel for taking your time and great conversation. And I think we're moving on to the next topic. So thank you. Thank you.