Beyond Chatbots: AI for Cross-Channel Media Optimization | Amazon X AI Working Sessions

Good morning, everybody. I see we've got more attendees starting to stream in for our second to final session. Uh, really excited to see all of you continuing to pay attention. Thank you so much for your time, uh, and dedication to hanging out with us. If this is your first session with us, as a reminder, I'm Andrew Hamada, co-founder and CEO of Reason Automation. And we are going to be hearing from Juan here in a second from MuttData. Just a couple of housekeeping things. If you have questions, you have the ability to use chat here in Zoom and talk with either us or with each other. But there's also a Q&A tool that you can use to ask questions anonymously if you would prefer. With that in mind, Juan, over to you. Would you like to do a quick introduction, please?

Hey, Andrew. Sure. Great to be here. Happy to join the different sessions. I know you had a great lineup. And from our side, I'm Juan. I'm from— I'm MAD Data CEO and one of the co-founders. We are a data and AI services company with a significant focus on martech and marketing and retail media use cases. And we work all across Latin America and the US on this, mostly with enterprise companies and some mid-market, right? So today we are going to focus on something that we worked a lot about, that it's going beyond like chatbots and just like querying and focusing on actually driving and increasing return on investment and marketing actions. And this is what the talk is about. So starting with it, what we are looking to talk about during this next 45 minutes, it's basically 3 things. One, like why we think in terms of optimizing your digital marketing spend and you're getting better return on investment on that digital marketing spend. The already in-place platform optimizers and intelligent campaigns are not enough, why you should build upon that. This is a more particular problem that we show how we can approach it with machine learning, statistics, all that. But then this gives us— illustrates a way to actually give context to agents that can later decide on, right? And this is about like building the right foundations and context for these agents, right? For them to take actions. How saturation curves help with this. And we're talking about like mathematical models, but in the end it's pretty intuitive and it also helps any marketing team and any, any people in general that are investing heavily on marketing, but especially on digital marketing to actually understand where they stand in terms of saturation and optimization opportunities of their media investment. And then what a complete marketing intelligence platform looks like in terms of the different layers, right? Like the most we're going to see in the first two sections actually are part of could be part of a broader platform that can drive the marketing team's processes and help it take to the next level of the marketing operations, but the overall marketing function, right? So a little bit of why we are talking about this. So again, everybody knows that we are in this agentic year or era already. And you're going to see that a lot of the marketing platforms are already saying they use AI, they optimize with AI. You don't need to worry about like running your own thing, like just use our, our agentic layer, agentic campaigns or whatever. And that's going to be enough. But for sure, like we found that for several reasons it's not enough, right? One of it is in terms of like you're doing the same as everybody's doing, right? In terms of using the same campaigns and staying competitive in that way is not the best option, right? But also you have to keep in mind that every platform basically optimizes what they see, right? In terms of the signals, the campaigns you're running there, and it's hard for them to really decide and also give you the signals that basically it's better to spend the additional money elsewhere, right? So basically what we see is that again, running campaigns nowadays is a lot easier and much more efficient than running it a while ago in terms of that you don't need to take care of that many structures and you don't need to build like decide on a CPC for each of the keywords. That was a long time ago in terms of search campaigns, right? But even though we have that, like when you look at the problem from the advertiser's point of view, like the advertiser doesn't care that Google does the best job and Meta does the best job with their budget. The advertiser cares to maximize the overall results across all of the channels, right? So in the end, like if your best results are obtained by moving, I don't know, half of the budget from Google to Amazon DSP, then Ideally, you want to have that understanding and that answer. And ideally, you would love for that to be driven automatically or at least with some human in the loop, but with the possibility of running it automatically, right?

So this is great because it's almost like you listen to my customer calls and my sales calls and you heard a very, very common pain point being expressed. The way that we hear it is, you know, how do we spend our next dollar? You know, one when we have this huge omnichannel advertising strategy that's both on Amazon and off Amazon, includes many different campaign types and different optimization strategies and keywords, it's almost impossible to figure out how to optimize it across all of those channels. And to your exact point, everybody says optimize it within our ecosystem, not across it. So I promise everybody, I did not help him write this. This is entirely a coincidence that he is talking about this exact way that we think about it.

Yeah, sure. And there's even one title that basically talks about where to invest the next dollar. It's a bit further down the presentation. But yeah, I was going to say just that. And from our point of view, we really think that what traditionally worked in terms of doing some manual analysis and optimizing whatever every week probably ideally, or even every day. Like, once you have like several campaigns, it's really complex for the marketing team to run that and run a proper analysis. And they usually move from daily to weekly and then to biweekly, and you lose a lot of opportunities to adapt to how the market changes based on that. Then a lot of the platforms have some rule-based automations, right? And that's helpful for a lot of situations. And for sure, I would tell teams to use that and set up some basic rules in place. But it's really hard to really use that to anticipate most of the common scenarios or really understand like the typical, okay, if I take $10 from this channel and move it to the other channel, I would obtain like double the results, right? Like It's impossible to do that with like simple rule-based automation. And then as we mentioned again, like many companies are talking about how they have incorporated these AI campaigns, automated campaigns or whatever. Usually they ask you to just create one campaign, put all your money in it, create all of the ad groups and leave all of the optimization to them. But that again optimizes locally. And if you've read any of the papers that Google or any other platform has put up, like they're not going to do the best for every advertiser in every opportunity, every auction. They are trying to maximize their results complying with your targets, right? But maybe you could basically, I know, increase your ROAS 10% if Google had favored you in several other auctions. But without automated campaigns, you're not going to be able to actually manage that, right? So basically, there's a word of caution. It's a word of caution in terms of using these campaigns indiscriminately, because basically, if you have a small team, you have a lot of campaigns, and you have a lot of I know, let's say categories where you're running and there's a lot of complexity, then probably you can begin with them. But in the end, like once you are really trying to drive that performance to the best it can be, like these campaigns are not going to get you there, right? Okay, so with this in mind, thank you, Andrew Hamada, for the preview. But basically We still think one of the hardest questions for any like significant, like large-scale advertiser, any large e-commerce companies having several brands, all that is this budget allocation, right? And like many people think about it in terms of like what do we need to do for our next quarter or our next week or whatever, right? But in the end, like the budget allocation is something that's done in real time. Like ideally if you if you would know the perfect combination like every minute, you should probably switch your investment between campaigns and channels like every minute. It might not be that feasible, right? But it's really complex to understand how several factors like drive or change this performance in terms of like you need to continually monitor that. Like again, if you're running a proper operation, then probably you're going to have some stability in terms of results and you extrapolate that to the following days, right? But again, like the market changes the whole time and ideally you would look every day or even maybe several times per day to what actually is happening. Most marketing teams actually do this on special dates, right? But it's hard for them to keep up with it. Then there are like a lot of different variables that when you see teams like deciding what to do with their next marketing dollars, right? They usually simplify the equation and generate some, like, more linear analysis in Excel, right, or spreadsheets, whatever. And that basically doesn't take into consideration, like, several other variables that might be affecting your results. And finally, it's really hard to understand with these tools and with just looking at the different platform dashboards, like what's the right level of saturation and elasticity of the different campaigns, right? So it's really hard to understand like this typical thing, like if I took 10% of this channel and move it and distribute it, how should I distribute it to the other ones, right? And if it's the right call or not, right? So in the end, this is a complex problem to solve really well. It is easy to have a good enough basic version, right? In terms of like with the— how the marketing campaigns have and the platforms have progressed, you should probably have something good enough to start pretty quickly. But in the end, like when you're really working with top brands and with really significant digital marketing spend or spend in general, right? You really need to see how you can drive that additional 5% improvement in ROAS. And that's one of the big questions there, right? So what I wanted to bring is probably something that a lot of people have seen in the past. This is not news and this is something that it's a tool, a mathematical model basically that relates your media investment, right, to your campaign results, right? So let's say for a particular campaign, we have this relationship where you're probably going to have the As you spend more in that campaign, you continually increase your response or your results in terms of sales, like GMV, conversions, leads, whatever you want to bring. And this usually creates this shape of diminishing returns, right? Especially like performance-based campaigns behave in this way. Where you're going to see that initially you spend the money and you get a significant return, right? Then like the campaign starts saturating, right? But you still get a positive return with every additional dollar you spend. And then basically you reach a saturation zone where probably your money would be better spent elsewhere, right? Like if you are spending last, let's say yesterday you did $100 of spend and you are already seeing this saturation zone, right? Probably if you're going to spend today $110, you should move those $10 additional to another campaign or another channel, right? If you can do it for sure, right? So, so we're going to talk a little bit about how we can build this. But before that, one thing to take into consideration is that many marketing teams talk about ROAS, right? And basically the relationship between like how much are you spending and what are your results, right? And if we were able to basically know the perfect shape of the curve, right, you would be able to understand at every spend level like perfectly what are you going to get in terms of revenue in this case as a result for your campaign, right? And In this case, you're going to see probably that you get a ROAS of 1.8 if you spend like $3,000. And basically, you probably in terms of volumes, like you want to maximize your marketing spend aside from getting a good enough ROAS, right? But what people like usually forget is that the ROAS measurement is actually an average, right? In terms of Basically from spending these $5,000, you know, on average I got this 1.2x return that it might show that it's good enough because you're probably like 20% return. It could be good enough for many of the campaigns or any many of the things you're driving. But in the end, the question when you have several campaigns and all that is not just like like what's the average ROAS I got for this campaign, but it's also what we call the marginal ROAS, right? So the marginal ROAS means basically that you might be at the spend level of $3K. Basically, you might be getting a similar ROAS in terms of both like the average, but also according to the saturation curve, you can see that the marginal ROAS is on the same level. But then again, if you keep spending more on this campaign, you're going to run in, in this saturation curve area where basically every additional dollar is basically generating a negative, you could say, result in terms of basically we are spending $1 and getting $0.30 in return, right? So basically, if you see this, probably there are some place where between $3,000 and $5,000 that you are going to run into the 1x marginal ROAS and that you shouldn't spend if you have other options, right? That additional money in this campaign and that should probably move it to another one, right? Again, we are talking about basically when you are spending one day, let's say $100, and you need to decide what you do in the next day, right? Like you need to understand, like, again, I have these— I may be having my budget $100 per day, but maybe I don't know. I also have some planning where like that varies according to day of the week or whatever. So if tomorrow I have $110 and basically my model understands that the saturation curve is the same, then probably I should move that 10 incremental dollars if I'm in the saturation curve area to another campaign, to another channel and all that, that basically is not as saturated in that spend level, right? Again, if anyone has any questions or Andrew, if you have any questions, feel free to ask.

No, I'm following. So far at least, I think you're doing a great job of making a fairly dense subject comprehensible. So I'm following and I don't see any questions so far. Keep going, sir.

Okay, perfect. Yeah, that's what I'm trying to do. Again, just to reiterate, usually when people talk about ROAS, are talking about an average ROAS, right? So basically at that level of spend on average, we got this return on investment. But when you're running like your marketing operation every day, basically you should really think about the marginal thing, right? So basically, if I move the investment on that campaign, if I add $10 more or subtract $10 from the daily budget and move that money to another, like, is that additional $10 bringing any significant increase in the marginal ROAS, right? And so basically, this is the forward-looking that helps you decide what to do the next day, ideally. But then again, these saturation curves are not that simple to actually know, right? Like, no platform is going to tell you, or no marketing campaign is going to give you the perfect understanding of what the saturation curve is. So how do we try to understand, like, an estimation basically, and an approach to what could be the saturation curve for a particular campaign. This is a simplification for sure, but basically what we try to do is obtain several data points in terms of like, if you already have this always-on running campaign, probably taking whatever, 7 days, 12 days, 60 days, whatever worth of data you get For each of the days, you get basically one data point, right? That's what was the spend for that particular day and which was the revenue we generated based on that spend, right? And you plot it basically and you try to find the curve, right, that fits the best to all of the data points. That basically is going to give you like the estimated saturation curve for that campaign. Ideally, you would have other things that would move that saturation curve. Like, for example, let's say again tomorrow it's a Friday. If you like, you should probably weigh more the data points related to Fridays of the last 60 days, right? Because they're probably going to predict a little bit better what happens on a Friday. So there are things worth considering. But at the end, if you had for a particular campaign a varied set of spend levels and you have a varied set of results for each of those days, right? You probably are able to fit this and understand. And this gives you the tool to understand, like for example, if I haven't spent at this spend level, but I can give a good estimation if this was the saturation curve of what the resulting revenue would be. And I can compare that to other campaigns, to other channels, right? Like we can basically do this at the campaign level or the channel level or the asset level, depending on how you want to run things. And that gives you the general idea when you are actually planning how you can basically compare these different options and then decide on how to optimize them. Yeah. So, so basically that's the general idea of a saturation curve based on digital marketing performance, right? Many people are probably going to be familiar with the term media mix modeling or marketing mix modeling, right? That's another approach, right? Like another mathematical way of also generating saturation curves. Usually that's done more at the channel level and for a longer time period, like at least a year and a half or 2 years worth of data. What I'm talking about is actually how you can create these saturation curves for your digital marketing campaigns and use them on a daily or weekly basis, right? Like, you're going to see that whenever you are looking at the market, whenever you see a budget allocator platform or a campaign optimization platform, many of them actually work with these type of saturation curves., right? And this would give any marketing team that's running digital marketing structure like a way to use this. And also basically once you are doing this, you can do it for any channel, any type of campaign in those digital marketing channels. So you can compare with the same method, right, these different things, and you're not just relying on whatever the marketing channel says, right?

All right. So I'm sure you're going to get there eventually. But the question that I think I and everybody has is going to be how?

Yeah, for sure. So, so basically how you reassigned the— how you optimize the marketing investment. Okay. So, so this is basically the— what you're going to see again, this is an illustration, but basically like for every day you can see, okay, that day I spent this amount of money, right? And I got this amount of revenue. So basically you can calculate the marginal ROAS right there. That's, that's basically where you are in terms of the slope of the, of the curve. And if you plot this according to— in like by day for the last X amount of days, you're probably going to see something like this, right? Some campaigns have a higher marginal ROAS, some campaigns have a lower marginal ROAS, right? And in the end, you're probably going to be able to calculate across all of the campaigns what's the optimal one in terms of for all your marketing campaigns, right? So in the end, like if you have a higher marginal ROAS, you should probably add more money to that campaign, right? And if you have a lower one, you should probably subtract marketing spend from that campaign, right? Basically, above optimal, you are underinvested, you should spend more money. Below optimal, you are overinvested, you should basically subtract spend from that campaign or that channel. And if you're aligned with the optimal, that basically it's— you are okay and that campaign has the right spend level. That's going to vary again according to your dates and your seasonalities and many other, like if you have in terms of like stocks and all that. But basically this gives you like the mathematical tool. I know we're not getting into the weeds of it and we're not actually, I'm not actually telling you about how to calculate that perfectly. But with this in mind, like if you have all your saturation campaigns and you have your spend levels, sorry, saturation curves and your spend levels, you can run an optimization process. Basically, it's— you can use any Python library that fits this problem and understand what— how much you should move from each of the different campaigns, right? So what I'm getting there here is that, like, if you're able to build this and we have done this and we have a platform that does this every day for advertisers, We've seen really good results in terms of like for, again, for large advertisers, we have a larger advertiser portfolio in LATAM. When they incorporate this, they see benefits in terms of the marketing operations in general, making it easier to run and decide your budget allocation. But also you're going to see efficiencies at a certain spend level for sure. More than compensate any spend you're going to dedicate to building these capabilities, right? And these are some examples of the results we've seen. In some other cases, we've seen even much bigger results than this, but we try to avoid having people have their hopes up so much, right? But in terms of What we see is that the marketing operations usually have a lot of different problems, right? And doing budget allocation according to acceleration curves is just the beginning in terms of using mathematical models to optimize your marketing operations. Many people probably are familiar with better models that can more accurately attribute the incremental impact of each marketing touchpoint. Usually called data-driven attribution. Many people are probably doing some kind of demand forecasting, right? Usually if you use machine learning, like, it's a lot better, the results you're going to get if you use the right models, right? Conversion propensity is also an interesting topic, like really segmenting users according to this, like if they have a higher conversion propensity, you could probably create audiences for that, have a have the marketing platform bid higher for those, or even like just let the platform know this for it to bid automatically higher. If you have a better measurement of the LTV, that's also a really useful thing to send back to the marketing platforms, right, to the ad platforms. As offline conversion information. So they can really understand how to segment internally and when to bid higher. And there are a couple of other things that are related to machine learning or some typical processes that maybe they're not terms that really were known on the marketing front. But anomaly detection and root cause analysis is basically what every marketing analyst does the whole day, right? In terms of— like finding weird things in the performance of the campaigns or any marketing KPIs and then trying to understand why that is happening, right? If it's because, I don't know, you downloaded the data and the data wasn't properly processed, or is it because we, I don't know, we have some low stock quality depth or whatever for a particular category, or is it because there's an increasing competition or whatever, right? So, so that's the typical process. There are many models that can help you basically understand a lot better how you can generate from those models the information to actually prioritize anomalies, understand their size, and also understand what are the potential root causes if you have the information right. So ideally, once you are thinking about using any agents and any agents to actually generate recommendations on actions or actually generate actions on your marketing spend, like you should really think about building capabilities and models related to this. Again, many people think about when they think about the agents or using any AI chatbots, they think it's mostly about building context and that's the right approach. But many people think about context just in terms of I give the agent access to a particular table or the right data or I don't know, like a document for, with my marketing strategy, my branding, all that. And that for sure is necessary. But what usually happens is If you are, again, if you are doing any complex analysis, probably having access to the raw data is not enough, right? Like you probably would love to have, I don't know if you're going to decide on your budget allocation for the next month, it would be great to have the output of a demand forecasting model. And if the model says that basically this category is going to drive a significant increase in terms of demand, then probably you should spend more money there, right? So like many people need to think about in terms of like how they're building the context of these more refined and more refined analysis and as an output of the machine learning models, right? Because with that context, it's easier for the agent to take— to make a decision that's going to impact the future, right? Like if an analyst is better suited by a good demand forecasting model, then probably if you give that input to the model, to the agent, sorry, you're probably going to see much better results in terms of what the model is going to recommend. So we always need to think of how we are actually elaborating and generating a better context in terms of quality, not only in terms of quantity, right? Like you're going to see that if you send this like big context with raw data, like usually models are going to hallucinate much more. If they have a small table that says, okay, next month the estimated demand for this category is going to be this much and for each of the categories you have that, it probably is not going to hallucinate about that number, right? So you need to think about how you are aggregating and how you are generating that process data, right?

Yep.

So in terms of how we see like the most advanced advertisers work, for example, how they see, how we see them combine their different models to build a much better optimization platform. I can give you this flow of data where many advertisers are actually using their attribution systems, right? Like AppSlyr for mobile, Adobe Analytics, Google Analytics, whatever, to have their cross-channel tracking, right? Dumping that into their data lake or data lakehouse or data warehouse, whatever you want to call it. And then basically combining a bunch of models to get a unified measurement, right? So when we're talking about a bunch of models, like people usually always think about just like having the best attribution. Online attribution, I prefer to call it. So basically having a better data-driven model. Google Analytics already has the data-driven model. You're going to see, for example, Amazon recently released their data-driven attribution model for Amazon Ads, right? So you're going to see that many platforms are going there. What usually happens is when you are a large advertiser, it's really hard to understand why Google credits a channel or a campaign a certain way, right? In Google Analytics, you don't have the understanding of the underlying model. So what we see is that usually like big sellers or advertisers usually tend to build their own data-driven attribution model. But then usually if you are spending money offline and online, you should really think about how you can combine in a single model both your online attribution and your measurement— sorry, your offline measurement. And that's one of the reasons people usually try to bring in some kind of media mix modeling, right? If they have enough of a stable operation for at least a couple of years, usually we see many companies investing in media mix modeling, right? And trying to combine that. And the way to combine those two into a unified measurement is to also have some kind of ground truth, right? Probably not for all the channels all the time, but some kind of ground truth where you can basically say, okay, my media mix modeling is going to have some biases. My data-driven attribution is going to have some biases. I need to have some way of understanding like what's at least for one of my channels what's the baseline of incremental revenue it's bringing. And that's basically running a conversion lift or a geo lift test on some of your channels. Ideally, you're going to see that depending on the size of your media investment, it's going to take between 1 and 2 months for a particular channel. If you are spending nationwide, then probably in the US, then probably geo lift is the best option, right? And it's a It's probably the most expensive one, but it's going to be the most accurate one. If you're not spending nationwide and you have— and your spend is not high enough, then probably you can just run platform conversion lead for Meta. That is the easiest one, or with Google that nowadays they also allow you. And basically with those 3 data points, you're going to be able to calibrate and generate like a unified measurement. Once you have this unified, proper unified measurement, you should really understand much better the incremental revenue each channel is bringing and each campaign even probably. And then basically you have to optimize your media spend. And then you, that's the time where you are actually calculating your saturation curves and using that to drive your marketing performance. All of the time you need to be getting the data from the different platforms and then combining it. And this is one of the flows. And you're seeing how you can combine these different models and get a much better marketing operation and use your data capabilities to actually drive incremental revenue, or at least avoid spend waste for sure. But then we're thinking about how we can combine this into a single platform, right? That can basically use and give the data to the AI agents to help you improve your marketing intelligence. And that basically is our next slide in terms of what we see is you should really start by the data platform, right? Like integrating properly your marketing data across your paid media channels, your tracking system, maybe your CRM if you have a CDP, any apps or whatever, like whatever information that's relevant that could impact the marketing performance, you should really try to ingest and have it stored in a unified data platform. We're not talking about any particulars, but again, any of the cloud hyperscalers, any Databricks, for example, any data platform should be good enough these days for you to store the marketing data. Then basically you're probably going to start by building your metrics on some, what we call semantic layer, right? How you measure ROAS, your CPA, your customer acquisition cost, CVR, having that data aggregated by different dimensions, right? Countries, I don't know, states, whatever. Month, day, hours, etc. And establishing basically your most important metrics in terms of marketing performance across the board. Then, as we talked about, you should probably think about building some predictive models. If you're like really focused on digital marketing, then saturation curves for those campaigns are a good way to go. But it depends on— depending on many things, like you should also think about like how we can measure the quality of the new clients you're acquiring with lifetime, predictive lifetime value, have some kind of predictive churn, forecasting demand and revenue. Once you're able to build this, you should really think about how you can build anomaly detection and root cause analysis capabilities. Again, you're, you probably, maybe will have had this like, you might have 10 different— for a given day, you have 10, 20 things that are happening that you— that a team has found weird and you don't know how to prioritize them or you don't know how to dedicate a significant amount of time to— for each one to really understand what's happening below and what's the underlying thing. There are models that can help your team with this, both in terms of understanding the anomaly impact or size priority to prioritize them, then understanding what's happening below to actually try to explain what's the anomaly. They're not going to be perfect. And with all of these, you're probably going to have enough context to actually bring in the agents, right? I'm not saying you need to build everything perfectly. And probably the best thing is to just start with some data integrated, some saturation curves or whatever, some small models and some of the basic metrics built and then bringing agents, right? I think people should work iteratively. But again, once you bring in any, any agent, any copilot to generate reports, summarize summaries, generate alerts, if they have all of these or part of these model predictive models, these well-built metrics and anomaly and root cause analysis, models, they're going to have a much better— you're going to have much better results in terms of the level of depth they're going to give you, the type, the recommendations, how well they actually perform. And that's when we are seeing, right, the real value added of having an AI agent actually generate the recommendations. It's not just like chatting with the AI agent, right, to to get a report is actually having the agent recommend actions. And then basically, if you have the right integrations, just execute those on your marketing data, right, or campaigns. So basically, so some questions you might want to take back to your team, to your company, and that we would love to talk about further with anyone that wants to. Like really think about like what's the marginal ROAS of each of your campaigns, try to understand it, try to do some analysis at least manually, then think about automating it. For sure, we've seen some significant results, especially with market-leading brands, right? They usually have like heavy saturation in some campaigns that could be moved to a new channel. Then how we can better measure and optimize in a cross-channel way, right? Again, many companies are using already some kind of tracking system. Most of them are using Google Analytics. But that's not all of the, the answer, right? And probably, you're not going to be able to understand what it does perfectly. So really think about if you're at the right level of complexity and investment to do something that's incremental and that could bring that— maybe doing something custom over your data platform could bring you a much better understanding and levers to actually optimize your results. And basically, like, how are you building your Agentic Marketing Intelligence platform? Again, we think that's the way things are going, right? Like you're going to end up building something similar to that in the end. So this is kind of the blueprint where many companies are going to land. They could land in 5 years, 10 years, or 6 months depending on how much they're investing and how focused they are and if they understand, like if they're building the right capabilities, right? So that's the end of the content for today.

Awesome. Yeah, super dense, but this was exactly what I was hoping for, sir. You know, when we were originally talking about this session, you mentioned, I really want to go deep on, you know, really high-level concepts and then drill down into specifically how it can be used in the optimization question. And I think we exactly nailed that. One question did come in very specifically about your marginal ROAS methodology. For marginal ROAS, the change metrics need 2 data points, so you can really only ever compare versus previous month or year. So when you're doing that, do you use a fixed or a weighted starting spend or revenue metrics to give the marginal ROAS more relevance versus just a comparison?

So usually, yeah, let's just maybe move for the Usually we're talking about campaigns that already have some kind of history, right, in terms of their daily spend. Again, the question actually asked about what you need to compare versus like the previous period, right? We usually for digital marketing campaigns run like daily touch, daily data points, right? So these points are going to be daily. So, so basically you for sure need to have at least 2 data points. Ideally, what we've seen is that some, like for most companies, we run at least a week, right? And then basically we start doing the daily generation of the saturation curves and the optimization, right? But it really depends. Sometimes you're just, you're just recreating the same campaign that you have already run with the same ads and whatever, and they have really similar performance. So basically, we can reuse a lot of the data points from the old campaign for the new ones, right? But in general, yeah, for sure, you need at least 2 data points mathematically. Ideally, you would, you would have like, I would say between 7 and 14 to start with, right?

Gotcha. Awesome. All right, one more question just came in, and I love this one because it's, I think, cuts to the point. With AI tools like Claude and MCB connectors and whatnot, what are the pros and cons of just using something like that versus using a more dedicated platform or set of tools?

It's a good question. I actually think that from like we've been using both things, right? And usually I think things are trending in the direction where Codex or Cloud Code or Cloud Cowork are going to be kind of the operating system, right? But you still need to build these capabilities in terms of the different models and the different aggregated data to get the right context to the models, right? So in the end, we didn't add— in this presentation, we didn't have any— we didn't add any names of which agents, right? But I think a lot of the work is going to be like maybe using cloud code and combining that with your sort of like metrics, semantic modeling, machine learning models, anomaly detection, and root cause analysis. And that probably is going to be better served by a custom-made or out-of-the-shelf platform, right? And that's why you are probably seeing all these, I know, connectors, right? In terms of like from Cloud Code or Codex, right? That you're able to connect to several like SaaS providers. And we think that that's because in the end, like you could build your own, I know, email client and basically have Cloud Code build it. You probably have a much better experience if you use Gmail and you connect Cloud Code to Gmail, right? And maybe you decide on how you to process your email through Cloud Code, or you get a lot of the data from Gmail and then basically you run some automations. But again, the same way the marketing analyst is not going— is going to have a hard time just accessing the raw data and doing from the ground up all of the work to generate all of these models, you're going to see that any agent is going to have a hard time. And I think it's going to be the same for a while. Like you maybe could spend, I don't know, a lot of money running the agent to generate all of this, but having consistent results and having it actually take action on itself when you're not in the loop, it's not going to be feasible if you don't like build like a mid-layer of processed data, right?

Yep. That makes a ton of sense to me. And it's very similar to, you know, as a data pipeline provider in the modern era, a lot of people ask me, well, can't I just like build this on my own with cloud code and, you know, stand up my own API adapter? And the answer is absolutely yes. Like you can. Every single component of what I do, what Reason Automation does, what Juan and MuttData do is achievable by a human somewhere with a tool. Like nothing here is like completely novel and unique in the world, but it's a question of where do you want to spend your time, your energy, your expertise, expertise and your enthusiasm. To Juan's point, like, the fact that you can make a CRM in 2 hours with Vibe Coding and stand it up and whatnot does not mean that that's your happy place to be maintaining a Vibe-coded CRM for the next 50 years of your life. And if it is, then maybe you should be building CRM software. But instead, our guidance is generally, if you are passionate about something and you find yourself in a workspace or in a tool where it feels fun. Like that's the stuff where get out of your own, or like get, like don't pay somebody else to do that work for you. Do that work yourself because that's where you're gonna get like that perpetual motion machine benefit of constantly wanting to do more and investing more and being excited by the challenges and whatnot. I personally have found myself staying up really, really late at night working on specific work streams in AI because it's unlocking creativity and passion for me. Versus others like editing, writing, that it doesn't make me excited to do at all. It just helps out, right? And so from my perspective, those sort of unexciting tasks like editing and writing are things where I have sought out tools, companies, and third-party providers to help me do it better rather than begrudgingly forcing myself to do it through, you know, manual processes and AI-assisted processes.

Yeah, and to tell you the truth, we've had 3 cases working with large companies that they have told us that they were going to build like a similar platform internally, right? One of them was before any agent, agent decoding. They spent a year and a half trying to do stuff. They were never able to do that because that's not their focus, right? And two of them after. And from what we've heard from them, they're having the same issue, right? In terms of like you're probably going to have a couple of analysts that can do this part-time. And that's also why, again, when you see companies like Uber and Amazon, they have the engineering knowledge and also the size of the teams and to actually dedicate 6 months, 1 year, 2 years to build these type of platforms, right? But most of the companies are not going to be in that area, right? And I think the same happens now. Like even if you can buy code some stuff and use a little bit of this, like you're probably not going to rely your whole marketing operation on something that's buy coded or you're going to have to spend a lot of money and time also to actually take that to a good engineering level and stable platform, right?

Yep. All right, one more question came in, probably a restatement of the previous one. How do you approach full-funnel multichannel measurement when the underlying metrics don't align across channels? So that makes sense. For example, ROAS is straightforward to attribute for search campaigns, but much harder for DSP where there isn't necessarily a product level or sales level attributable thing.

Yeah, in the end, what we see most of advertisers is to rely on something similar to this with another one additional level that was adding a little bit more complexity that was basically some causal inference analysis, right? So there are some methods that with causal inference, usually you're able to, measure like more point-in-time campaigns. Like let's say we're having a huge launch one particular day, right? Right. MIA mix modeling is not going to be the best one, best method for that. And if it's a live one, right? So basically, you're going to try and combine a bunch of measurement methods. What we're seeing is that usually if you combine MIA mix modeling, if you combine data-driven attribution, if you have some like GeoLift and some causal inference, you're going to be able to have good enough measurement across the board, like for full funnel campaigns. The part of it is actually you're going to see there are many papers nowadays on how to do this combination of the different methods, right? But there could be just simple multipliers that basically you get from the lift test, you get some multipliers and you basically use some of the Miamic's modeling models capabilities to to align or calibrate. That's what's called the, the MIA mix modeling to those multipliers. And then also basically you use those multipliers for the data-driven attribution and that gives you a unified measurement. But all in all, for most methods that are not the digital performance, digital marketing campaigns, you need to have a good enough history, right? In terms of maybe a year, 2 years worth of data. And that basically is a prerequisite. And it shouldn't be like if your business and the overall market has significantly changed in the last year or so, then probably you're not going to get the best results from me and mixed modeling, right? So overall, there's no silver bullet. You need to combine several methods and that's going to be like a pain we're going to live with for the whole time, right?

Gotcha. Juan, thank you again for your time. I really appreciate you taking the time to walk us through some very complex but very, very critical concepts in saturation curves and how they might apply to not only Amazon Ads but omnichannel marketing in general. Thanks everyone, and have a great rest of your day.

Thank you, Andrew. Good to be here.