https://www.listennotes.com/e/c4c1ef79a01542e7ba93958ec27ab9ee

Hey everyone. Welcome to the Intelligent marketer. I'm Mike DeBeau, partner at Greylock.

And I'm Rishabh Jain, co founder and CEO of Fermat. This is a show where we talk about the intersection of AI, marketing and commerce, what's changing, what's working and where things are headed.

So we're going to bring in guests and share what we're seeing in the field and try to make sense of how marketers could actually use this stuff. Last week we brought in Eric Seufer, a good friend and favorite of Rishabh and myself, to talk about the next iteration of his Everything is an ad Network thesis. We also covered the future of marketing and ads and LLMs, something that will to explore with future guests. So thanks for hanging out with us. Let's get started. All right, Eric, thank you for joining us. You have the honor of being the first guest on this new series that Rishabh and I are doing around kind of AI and marketing. And for those who don't know Eric Suefert, he's probably who I believe to be the most insightful analyst, writer, thinker on all things advertising and really kind of marketing related. He writes Mobile Dev Memo, which is one of the best know sources of content out there on all things advertising and specifically kind of mobile advertising, but much more than that nowadays and invests out of Heracles capital and is an advisor to many successful companies on their ad strategies as well. We've had Eric on the podcast two years ago to discuss kind of the future of marketing in the post privacy era, the post ATT era. And I felt like a good place to start would be, you know, going back to that. So when we spoke two years ago, we talked about how everything was changing now that ATT is a thing and we're kind of squarely in this post privacy area. We discussed a few areas where we thought the ad stack would be transformed so maybe we could look back at those and see where we were right or wrong and kind of what marketers should be planning for going forward. I think one of the first areas we discussed was measurement and I think one of the biggest challenges that marketers were facing at that time was this direct attribution model that we had all been trained to had kind of broken. And some marketers are scrambling to test incremental lift and build more probabilistic methods, maybe just jumping right into it like via measurement or otherwise. I would love for you to just comment on if you think about kind of the era of post privacy marketing over the last couple of years, what's been most notable and what kind of themes come top of mind to you?

Yeah. Well, first of all, thank you for having me. It's an honor to join you to be the guest on the first episode. Appreciate it. I think that measurement was the thing that changed most meaningfully as a result of att. And what's really interesting now is we're experiencing what I've called the privacy shuffle. Right? So we had this period where the privacy parameters around using consumer data for targeting and for measurement, those parameters kind of became much more restrictive. And I think there was generally a sense that, like, we need to prepare for the worst. Right. So, like ATT was pretty bad, but it could have gotten worse. And my sense was like the kind of prevailing wisdom was that it will continue to get worse. Like this is just going to be like a regular drumbeat of restrictions. And my sense sense is now that is not the case. The prevailing wisdom isn't there yet, but I think it will be in six months that actually we've seen the worst. There may be actually even some capitulation. And there has been actually with Google, you know, kind of doing a 180 on cookie deprecation, I think they're probably not going to deprecate the gaid now. So we have seen a little bit of stepping back from the edge of the cliff. But measurement did get overhauled kind of utterly. Right? Like fundamentally as a result of att. And I think what people are realizing is, is that, yes, that was catalyzed by ATT and like other privacy restrictions, but it would have been a good idea even absent ATT and absent those privacy restrictions. And everyone's kind of glad that they did it. And that actually stays with us, even if we do see kind of a privacy shuffle and there's less of a priority on privacy and that we're probably at the end of any sort of, like, big radical platform privacy changes. Everyone realizes that, yes, making the change to probabilistic measurement was beneficial and it was productive and they probably should have done it sooner than they did.

Yeah. So I think this is an interesting thread to Stan, and I think rather than kind of us geeking out on how to execute lift, testing the best, or building mmms, I think what's interesting is what are the second order effects, like when you actually have a good sense of lift and kind of look at measurement probabilistically versus just kind of the attributing all credit to some last touch or some direct attribution. Model? How do marketers actually make changes accordingly? Who are some of the winners and losers of that game, Eric? And maybe how do you see that continuing?

I think everyone's a winner. I think everyone's a winner. So this is not zero sum in any way, right? It's better measurement. And because it's better measurement, advertisers get to invest more. And if they get to invest more, they probably get to invest more everywhere, right? Like, why did Facebook put so much money into Robin? Why did Google put so much money into Meridian? They ostensibly they would be the benefactors of Last Click attribution, right? But why were they so invested in getting these tools into people's hands? Because they know that if everyone's deploying money more wisely, they benefit even if it's not. Even if they don't benefit the most, they still do, right? And so, like, I think there are no losers if measurement gets better broadly. I mean, other than like kind of fraud sources of traffic, but I think like the legitimate sources of traffic that provide incremental value, like, they will win. They will win from this. And it's just gradations of winning. I wrote this article a while back called Flying Blind, right? And it was actually the title of a keynote that I was giving for a while. And the idea here is like, you know, I had this idea in the keynote of like, Wittgenstein's ruler for ads, right? So, like, that's a concept from Nassim Taleba, invoking Wittgenstein. But he was saying, if you're using a ruler to measure a table, you're also using the table to measure the ruler. And I think what a lot of people were getting wrong about deterministic attribution was like, well, there's always just sort of like a little bit more sophistication I can add to it, and it'll be pseudo deterministic. I can always get there. There's just a little bit more voodoo I can throw into the mix and I'll be able to get to pseudo deterministic. And, like, what that tells me about them is they have no idea what they're doing, right? That's Wittgenstein's ruler. If you tell me, like, no, we came up with this very convoluted, complicated system for achieving pseudo deterministic measurement. It doesn't tell me that they're really good at measurement. It tells me they have no idea what they're doing, right? It tells me more about them than it does about their measurement apparatus. And so I think like when you let go and you, you sort of like appreciate all the capabilities that are opened up to you. If you fly blind. You've got podcast advertising, you've got TV advertising, you've got CTV advertising, probably the most explosive, exciting growth vertical in digital ads right now. You've got out of home, you've got. Everything can be measured if you shift away from deterministic.

Where do you think we are on that shift? Out of curiosity, what are you seeing in terms of the change management expectations that people are seeing? Which segments are adopting faster versus slower? I'm pretty sure not everybody's there. No, but it is clear that we are. We moved really fast. And so I'm wondering like where you think we are on that adoption cycle. What is a set of things that need to happen for us to get further up that adoption cycle or accelerate that adoption cycle?

We're still, I think, in the early adopter phase, but I think what ends up kind of dragging everyone along is just the platform tools getting better. And by that I mean Advantage plus and pmax. Totally like you get Advantage plus and pmax, you need this kind of marketing measurement for just Advantage plus. Like when you're throwing money into Advantage plus, you're not just buying newsfeed, you're buying everything. The metrics are obfuscated, right? You need this kind of measurement just for Advantage plus. Even if you're only working on Meta, only working on meta's properties, you need probabilistic measurement for Advantage plus. Right? And so I think what ends up happening is A, like the platform provided tools get better, but B, it's everyone recognizes, look at 50% of my budget or 80% of my budget on Meta, which is maybe 40% of my budget overall for E comms, higher is going into Advantage Plus. I need this kind of measurement. I just can't rely on last click. It just won't work. But I think we're still in the early adopter phase. The other thing is like, look, the point I've been making for a long time, everything is an ad network, right? Well, that's retail media, that's ctv. You can't measure that with deterministic. You just can't. It's impossible.

So I think this transitions into one other trend around post att, which is leveraging your own zero party or first party data to drive better targeting in a world where you actually can use, where there are less kind of signals that you can actually target upon. What are you observing there? So obviously retail media is one thing. But what are some of the smartest teams doing on that front right now? And maybe what again maybe jump into second order effects, which new networks kind of most benefit from.

This at the frontier is actually doing signal engineering. It's building the best possible, the highest density value signals. It's making sure that what you get back to the platform is so valuable that it sort of like disproportionately drives campaign optimization. The thing is like I think people look at a cappy and it's like it's a machine gun, right? It's just shooting everything over. Would you rather be sending 50 signals per user or would you rather send one really good high density, high value signal? And I think you'd rather send the high value signal and then if. Because it's also like a function of the quality of the inputs impacts the quality of the outputs, right? So like if I'm sending like valid, you know, sort of like highly credible meaning like the sort of like predictive power of that signal towards making a conversion as high. And so if I'm able to provide better inputs to the platform, they're able to do better prediction. So like it just sort of this sort of impact the benefit compounds throughout the sort of like the model. So that would be at the very cutting edge. And then I think sort of slightly back from the very bleeding edge of the frontier is just implementing CAPI and make sure that you're able to do that, right? And so that's actually more challenging than it sort of seems like just implementing a capi is no big deal, but implementing multiple cappys is fraught, right? Because I mean you have actual data leakage concerns, right? And so then you're talking about maybe working with like a CDP or some sort of like intermediary, which takes work, right? That's. And it's engineering work. So it's like, you know, real work, it's not marketing work, but like, so that is, that's more like organization. It's an organizational challenge, but you know, nonetheless it's very valuable. And then I think, you know, the other piece of this like that I'm really excited about when I think about like the applications of AI and I wrote about this today on LinkedIn. It was like a shower thought kind of thing. But like if you think about the applications that are the most exciting, it's actually like the down funnel stuff. It's actually doing like personalized economy stuff. And that is a user acquisition function, right? That is a marketing function. Like if I get the monetization right? If I get the economy planning right, all of that flows through to user acquisition.

Say more on that. What is personalization economy? How could you translate that?

So like the canonical hypothetical would be just a perfectly personalized price point or a bundle, right? And like what's really exciting now is that's more possible today than it was a month ago with Apple being found to be in contempt of this injunction order. So let's say I can put some into my website, right? And I have a very good idea of like their willingness to pay, their propensity to pay and the value that they get out of like multiple. Let me use a game just as an example because games are like the sort of easiest toy examples because everything is made up, right? Everything in a game economy is made up. There's no marginal cost of production for a virtual in app good. I can make anything I want be in app good. It's just the best toy example. So let's say that I've got a very good idea that a user has a high propensity to pay and they get a lot of value out of the sort of virtual in app goods. I get them to click out to the website. I have total sort of agency over my website. I don't need to register bundles, I don't need to manage price points. I have a total freedom to make anything I want and offer it up as a product. So now I just offer this person a $50 bundle of a sword, a shield, 50 gems, some gold coins and eight lives. Right? Now the next person comes around, it's just a sword. The next person comes around and it's 10 coins, but they're heavily discounted. I can personalize not just the price point, but I can personalize the product at the price point that I think has the highest probability of converting. There is a lot of power in that. And that is possible as a result of A, Apple having to give up the commission that's charged on the off platform payments. But B, just this application of AI, right? That's all just optimization. Like if I, you know that's a neural network powering that, right? What is the optimal thing to show this person to get the highest potential possible conversion rate. That is super exciting.

Yeah. It's a very important point that of all the things that marketers have been trained to test and optimize over the years, creative messaging, content, website formatting, et cetera, like pricing and bundling has kind of been one of the last frontiers. And as investors we're seeing more interesting innovation Here as well. And it's cool to hear you comment on that.

One more point is what was one of the first things that Spotify did when Apple was forced to adhere to the rules of the injunction? They offered bundles. They offered, you can now buy audiobooks. You can buy above and beyond the sort of amount of free time that you get with the subscription. You can buy audiobooks, add ons.

Exactly.

That kind of bundling is so economically powerful.

No, I was actually literally going to say the same thing. Is that actually it turns out that bundle offer merchandising is actually the hardest thing and is actually the thing that actually drives the highest ROI in most marketing funnels. It's just ridiculously painful to implement. Well, I agree with Eric completely actually is that AI for the first time might make it not ridiculously painful because if you were to talk to a merchandiser, whether it's for virtual goods or physical goods or whatever it may be, and tell them to do this, they would just start laughing. It would just be a ridiculous proposition to ask someone to do that. And then in the physical good world, it then has the added complexity of you then have to tell your 3 PL to do that. So yeah, I think you chose the perfect example with all virtual goods with this Apple injunction. Yeah, I'll be curious to see who starts building those systems because somebody's going to. And it's going to be in like the next year, basically.

Yeah, I think. Well, okay, but let me toss out my hypothesis here, but I've been, I've been so knee deep in this stuff. This has kind of been dominating my, my kind of thinking for the last like week. But there's a paper bundling and competition on the Internet. It's Bakos and bjorn Jolfsson. So BjornJulfsson is like one of the thought leaders in like digital products pricing. So like pricing for goods that have zero marginal cost of production. He and Hal Varian, like they kind of established that. A lot of that thinking there. But okay, so now Rishib, humor me. What if Advantage plus had landing page optimization? What if I had some kind of. There was some kind of API right now they were serving the ad, they got the click, they registered the click and then they were optimizing my landing page on my behalf. That's feasible from a technical perspective. That's not infeasible.

That's for sure going to happen. So not only is that going to happen, the inverse is going to happen too, which is like people are going to observe the shopping pattern and then Generate the creative.

Right.

So the system has to talk to each other in both directions. For sure that's going to happen. There's only two questions, like everything in platform life, which is who builds it, how does it get built and how do the signals go back and forth with the platform and the actual checkout? That's it. Those are the only questions because ultimately when the transaction happens of a certain type of bundle, the platform has to be able to read that information. Today I can tell you that Meta does not know how to read it. So what we're doing is we're actually saying, hey, platform, you need to learn how to take signals that are not identity based signals. Like we are now talking about skew signals and I've had many conversations with the Facebook API team. The only thing they know how to consume is identity based information and that's a huge miss basically. And so they're going to have to be able to get more, more effective at consuming new signals. But that is like for sure going to happen.

Or they need to acquire firmware.

No, Mike's going to have to walk into his partner meeting and say like, guys, we have risk of this company that's going to get acquired. Like, you know, sorry, Mike.

Speaking of signals and different kind of signals upon which, you know, marketers could target on, I think we should shift gears and talk about marketing on the LLMs. And this has been a very hot topic both in Ventureland and with brands right now. I think very quickly LLMs are where the, when I say LLMs, kind of like chat based apps launched by the larger model providers are kind of where the customer discovery journey begins. And I think we should talk about the implications and kind of opportunities that fall from this. And so I want to talk about a few different themes here, but maybe just to start, a lot of brands are now scrambling to figure out how they optimize their presence for LLMs. I know, Eric, you're more of a paid guy and this is kind of broaching into like SEO kind of land. But I'm curious for your view on this and kind of what, what you're observing there on brands trying to go and influence these results. Maybe just talk about that for a minute.

I feel like Eric is about to go onto the mountaintop and scream everything is an ad network. But I'll let.

Well, we're going to get to that. We're going to get to that because I think these are, these are inevitably ad networks. But, but before that, in today's world, before these, before platforms actually have perplexity as an ad offering. But, like, early right now, kind of before then, like, what are you seeing here?

Yeah, you stole my thunder. It's not going to have as much impact now. It's an interesting idea. I mean, the feedback loop is pretty slow there, right? I mean, if you're talking about pushing out content that will ultimately be trained on by an LLM, I mean, the kind of turnaround time on that is probably like months. Weirdly, I get a lot of. I get. Well, not a lot, but I get a decent amount of traffic from ChatGPT. Even though I've got ChatGPT excluded in my robots TXT file, someone actually sent me a screenshot. They were talking to ChatGPT and they were asking, like, about, you know, my view on something. And ChatGPT was referencing my articles behind the paywall. And so the person asked, like, are you a subscriber? And then ChatGPT, like, it was like just the most weaselly response. Well, I'm not actually a subscriber, but I can infer what his opinions might be from the headlines or something like that. Okay. They clearly found a way around the paywall or something. But I think it's an interesting approach. I mean, I think where I think there's probably the most value right now is just trying to find a way to get your content surfaced in AI overviews in Google search. Like, that's just kind of the existing approach of like, SEO and just adjusting it for this new paradigm of Google search. I mean, getting your results in ChatGPT is great, but I think AI overviews just drives a lot of traffic. And we know that because we know that publishers are screaming about all the traffic they're losing. Right. And so the companies that have, you know, sort of cozied up to Google and they've done data licensing agreements are getting that traffic. They're the beneficiaries of that. Right. And so I think generally, if you're talking about the open web, you're probably talking about. I mean, I hate to say this because people get upset. Right. But it's a dying paradigm. Right. You probably have to just do whatever you have to do to persist. Right. And what that is now is just making sure that you get Surface in AI overviews.

Yeah. So maybe let's get to the fun topic then. On everything is an ad network. And you've been right on that many times over the years. In the last week, OpenAI hired Fiji, who hailed from Meta and then Instacart and kind of built out in house ad products there. I don't want to speculate on that, but it feels likely that over time they will work on something. And this is kind of just the inevitable. Once you've aggregated that kind of impression volume, it's the inevitable path for these networks. And I think the semantic context that you're sitting on is just such a rich kind of data mine for advertisers to go work on. It does feel inevitable for all of these. And so maybe how do you think these platforms are going to navigate launching their ad engines? And for marketers who want to stay ahead of that, how should they be thinking about it right now?

I think they would be remiss if, first of all, they'd be remiss if they didn't. And it is inevitable. Like, there's no chance that OpenAI doesn't have a scaled ad platform in four years. Right. The hiring in Fiji was just like a little bit too on the nose even. Like you can't. I mean, I was going to write about that and I'm like, why? It's just so obvious. It's like saying the sky is blue. Right? I mean, of course that's what they hired her to do. That's not interesting. There's no insight there. But. But I think they would also be remiss in trying to like, trying to engineer something that they feel like represents a new paradigm when the reality is advertising has existed for thousands of years and it's best when done in a legacy manner. Right. And by that I mean you've got display, you've got it tailored to whatever conversion data you have on that person and you're able to measure outcomes. And that doesn't require anything fancy. That doesn't require. First of all, I don't, I think it's a horrible mistake to even raise the perception that you might be curating the content to serve an ad and not to serve the best possible answer. Right. That will kill engagement. Right. That just poisons the well, that, that will make people think that they're getting advertised to and they're not getting the best possible information. The best possible way to implement this is an ad alongside the best content. An ad alongside the content that most fulfills that user's needs, that will perform the best.

Yeah, totally. So there was this one thing that came up. I was at like a brand dinner three weeks ago, just bridging a little bit from our SEO conversation to this ads conversation. Mike, you might find this interesting too, but the question was asked like, hey, how Are people trying to make sure their content is showing up in the AI overviews and things like that. And I want to say 7 of 10, roughly 70% of the room literally said, we'll just wait for the ads product to get released. I think brands, when confronted with this question of do we want to play the SEO game in AI search, they're just like the total investment that I have to put into the SEO game relative to an ads network. I would rather wait for the ads product to come out and then invest the capital into the ads network versus into SEO. I think that that's actually like a hard question for marketers, which is like marketers get graded more on what they can actually show has an impact to the return than in something that is harder to show the impact. Right. And so I wonder how you think about that as it relates to where would you choose to invest right now? Would you choose to invest against trying to show up in SEO? Would you say, like, hey, let's just do our standard SEO, that way we know what we're investing into and then just wait for the ad product because, you know, it's inevitable. Like, how would you decide how to navigate that in the organization?

Yeah, I mean, I'm kind of like a performance absolutist, right? So like, I've never really been a fan of like SEO or aso, which is like the sort of apps equivalent because you don't control it, Right. You're at the, you're, you're sort of like subject to the whims of any sort of algorithm change. I mean, your fortunes can change overnight as a result of that. I'd rather just be in total control. Right. And so for me, that's advertising. You know, with SEO, you're exactly right. I mean, like, you can show you can kind of use difference and difference techniques, or you can do sort of like lift studies or GEO holdouts or whatever to show the effects of like SEO. But they're like sort of like there's a long time horizon that you have to kind of consider. And how do you measure the cost side? I guess it's just the cost of the salaries or it's just to me, it's just I'd rather have a dial that I can turn to grow my revenue essentially with paid ads, and just have a model that, you know, just sort of measures that with as much precision as possible. Right. So like, my sense is like, that's probably the right approach also. You just don't know. I mean, like, ChatGPT might have some sort of change in policy around ingesting content. Right. They may have a change in policy around citing the source of content. Right. Like AI overviews. In the same way, like, and also with AI overviews, I mean, you're talking about data licensing deals that like how do you supersede? Right. And the most commercialized queries are probably going to be the most valuable to sort of use a data licensing agreement as leverage to inject your content into. Right. So like all of these things are not in your control. I'd rather just build an end to end sort of growth engine oriented around ads.

Eric, I want to go back to your point on the format and I very much agree that the biggest mistake these engines can make is just going and compromising user trust by influencing results in such a way that are I guess like unnatural or overly paid. I feel like we're kind of waiting for our Instagram ad moment here where people kind of realize like, hey, these ads are actually enhancing my experience when they're surfacing products that are very relevant to me. The ads are reasonably engaging as long as you keep ad load kind of within reasonable range. I would argue that it actually enhanced the user experience. A lot of the thinking that I have seen to date here just assumes that these are going to be basic text based search ads that are going to look very similar to kind of what we've all seen in Google over the years. I kind of feel like there's a new native unit to be created that we haven't yet seen yet. And I thought that would be interesting to get your take on and kind of hear what ideas you have. Like if you were to be in charge of ads at one of these large players and you were tasked with creating the right unit, that would be the new AI native unit. Like what would that look like to you?

Here's my Freebie to you. OpenAI I'm going to give this to you. You can have it. So there's a couple, right? So there's one is be full screen non skippable interstitial for every X queries on the free tier. So you do five queries. You gotta watch a full screen non skippable interstitial. That's the YouTube ad, right? Same. It exists, right? That's not new. The second could be just real time display insertions, right? So these would be similar to like Google shopping ads. You just have those in between blocks of texts as they render. The other is, I could just imagine a tower unit, right? A tower unit on the side of the results, right? Like, why not? And that could be totally divorced. So the thing is, the less related those things are, the more trust you have in the text, right? Because think about it like, if I'm seeing an ad for, you know, I'm going to Hawaii tomorrow, Hawaiian Airlines, and the text is about, what is Eric super talking about when he says Wittgenstein's ruler? Then I don't doubt those results are sort of delivered to me independent of the sort of behaviorally targeted ad. But the other thing here is, I think if you kind of think out like a couple years from now, like, I don't think that the primary interface is gonna be text, right? You can imagine, like, you actually are talking to, like an AI avatar, right, who's like, persistent and consistent. Or maybe you have like a cast of characters, right? And this is all video. It's all delivered to you like a YouTube video, but it's all rendered in real time. Now, if you have that, then you just have a standard YouTube interstitial. You have a standard YouTube cutaway. Again, that doesn't sort of dilute the perception of the content. This is a breakaway ad. I'm going to pause the content to show you an ad. That's how ads work best, because you need to sort of get the concentration on the ad.

Rishabh, what do you think?

I think that that's exactly right, mostly because also the behaviors for a lot of these chat interfaces, where you're going back and forth, is a lot of questions that are unrelated to commercial intent. And so you actually want to design for a system that assumes that a lot of the engagement has no commercial intent. So it's like, if I'm doing a bunch of research about, hey, what are the most interesting topics to talk about for AI and ads for an upcoming podcast, it's like, what are you going to show me an ad for a mic? You know what I mean? That's not the best way to monetize me. The best way to monetize me is a bunch of other commercial information that you have about me next to that. And so also because of the usage pattern, trying to have it be inside of the content is actually just like, not the best monetization play. Like, purely economically, it would be the not best monetization play. Yeah, I mean, I kind of agree with Eric. I think on the screen you'll have like a tower on the side, and then on the phone, I don't know exactly what it looks like, but the takeover is like, by far the best idea. That I've heard so far, yeah.

So what if we go down the tail and there's a couple of companies we've come across that are trying to build essentially a new two sided ad network between AI apps and advertisers. And I think if you remove kind of the big handful of players who are going to be building their own kind of in house rails, there's presumably a much longer tail of companies that are never going to be able to charge premium subscriptions that will have to be ad supported from day one, that are going to be perhaps chat based to start with, but haven't yet figured out the right unit. Like if you were to go be designing an SSP for them, what would that look like? Forget ssp. If you were to go design kind of in house ads and like the Rails for them and ideally a two sided network for them, what would that look like? Do you believe such a network should exist and that it's not owned by one of the existing kind of players?

Yeah, I mean the thing is subscription is a suboptimal way to monetize pretty much anything, especially something with as much engagement as a chatbot. Right. Like okay, so Meta stopped reporting ARPU by Geo, but it was the last time they reported was 70 bucks in the U.S. that's a quarter. Right. So like what's ChatGPT charging for the subscription? Right. I mean there's far more ARPU in ads if you can target them really well. Like, like Medicare. Right. So my sense is like just everybody should be looking into ads. It reduces consumer surplus. Right? Like it produces a lot of consumer surplus, but it generates the most money. Actually I should say it produces optimal unit economics is what I meant. And so like I think everybody should be looking into ads if they can generate that kind of engagement at that kind of dau. Right. In terms of building a sort of intermediary, I don't think it needs to look too radically different from what we know today. I mean you need a DSP and SSP and an exchange, right? The companies that can build this stuff themselves and build like self serve and have like an ads manager, they're probably going to get the best economics and they're probably going to sort of build the best and most like functional systems. But I mean if you just build the programmatic infrastructure for that, I think it functions pretty well. And I don't know that you need a whole lot more than what you have today unless the formats are just radically different. Right. But again, I just don't think it is a good idea to do like a sales pitch in an LLM result. So like I would, I would advocate against that.

Anyway, does that mean that Trade Desk and Applovin and other networks start to serve that publisher space or does a new company get built?

I don't see why fundamentally you need a new company. I think yeah, Trade Desk could probably intermediate that pretty well or like an Applovin for different context, you know, the sort of content specificity. But, but I don't know that it gets any like, especially okay, if you're talking about a tower banner that that's trades could do that. Right now there's nothing you need if that's serving on a website and if it's serving in an ad. I mean there's plenty of video ad networks that could do the full screen interstitial takeover.

Totally. I guess the only question would be like, do they actually have the system to take in the data to run the algorithm based on the content of the conversation that you're having in the actual publisher? Right. Like that's actually the only delta is like today the signals that those algorithms take in are like the problem with today's ad network is like 100% of them are based on identity based data signals and then they have some sort of taxonomy against that. But very few are based on conversational signals. That ultimately is sort of what at least like when Mike and I have talked about it separately is like I don't know whether the current systems can actually support that data signal.

Well, okay, so there's a couple things, right? Like one is if you consider that to be just like a contextual signal, right? Like okay, so that's a piece of context. There's ways, just semantic analysis that you could do that, right? Like there's ways that you could just parse the conversation and send some key, right, that maps to like some kind of sentiment or whatever, a topic. But the thing is like, you know, like Applovin for instance. I mean I think they're principally serving in games. They're principally serving against context. It's principally serving against the taxonomy of the games and the game subcategories, right. So I think they have ways of doing that now. But my point I guess is like I don't think it's a good idea. I don't think it's necessarily or the best idea to take the topic or the subject or the general tone or whatever of the conversation and use that to target ads.

Totally. But you don't think that is a format of that data that you would want to store somewhere.

That's.

I guess the question is, do these systems have the ability, just like you're saying, hey, take in a data piece associated with the person. Now all of a sudden that data piece is much richer because it's actually a behavior interaction back and forth that you use at a different time or from a different place. Do you use that type of data or do you not use that type of data?

Not only can you not use that kind of data, you wouldn't want to, you wouldn't want to give it away. You see these experiments that people post on X or whatever, they're like, hey, go to OpenAI and say like, based upon what you know about me, show me a picture. And for me it's the guy.

Exactly, exactly.

That's actually pretty, the result for that is pretty much the same for everybody. It's just someone looking at like eight screens, right. But like there was a really interesting one and it was like based upon what you know about me, like what are like the five top flaws that I could fix like in the next month that would improve my output, blah, blah, blah. And it actually, it was like really insightful. And the thing is like OpenAI knows a lot about me. I mean, I bet you and it can map what it knows to exactly the kind of like demographic features that people advertise against. Right? It probably knows my age. It probably, you know, could guess my net worth or whatever, might guess my disposable income, could guess that I have kids. It knows all those things. Right. And so like knowing that, then what would you want to do? Would you want to pass that along to some intermediary? No. First of all, you probably can't. You wouldn't be able to package that up in a decipherable way and pass it along. But even if you could, you wouldn't want to, you'd want to keep that. That's gold, that is worth its weight in gold, that information. And so you would, you would build your own advertising system on top of that. Now it's the companies that don't get that kind of engagement that can't harness that kind of data that probably would then need to rely on like a third party system.

Yeah, I think part of your point that I'm hearing is actually you could derive identity based signals from this conversational data too. And so maybe actually feeding it into the existing DSPS is not as problematic as kind of we thought. I guess, last topic on this and I want to jump over and talk about agencies For a bit. So Shopify, you saw their integration with ChatGPT a couple weeks back. Now it makes sense that if we are all kind of starting our commerce discovery journey on these engines, it makes sense to be able to complete it there too. What do you think are the next couple of steps there? Like if you play that out, there's probably a conversation to be had on what's the role of the website in general. And so this is both for Rishabh and Eric. I feel like both of you guys will have interesting thoughts on this. How are you thinking about this?

I think that generally speaking, I'll just allude back to this dinner again. It was actually a very insightful dinner from a couple of weeks ago. They said that traffic that comes from one of these answer engines is higher converting than any other form of traffic. The sort of obvious insight from that is that at a minimum the total amount of research that you can do and therefore the total amount that you know before you get redirected is now so much higher that the role of the site has fundamentally changed to one of a transaction versus one of information. And so it kind of doesn't matter whether the transaction actually happens inside of ChatGPT or not to some extent. Because ultimately if the person is already made up their mind on what they want to purchase inside of the answer engine, then that is actually the most important thing. So that's like at least my 2 cents is I think how we think about whatever are these other properties outside of the actual answer engine they need to serve. What is the new reality, which is people have a ton of information based on their interactions inside of the answer engine. And so that's my principal point of view. I'm curious what Eric thinks, by the.

Way, if that's the case, do you think Shopify still deserves to own the cart?

I'm going to let Eric answer so I don't get into too much trouble.

Yeah, no, I agree with Rishabh. I think it's like by the time the user's clicked on the link, they've been sold. So whether the transaction happens right there or it happens eight seconds later because they had to click through, they've been sold on that thing. I don't know. That fundamentally alters the sort of calculus for the website or not, I think. And look, I mean, I just hate to be negative about this but like it, I do just think that the sort of open web paradigm has probably already come to an end. Right. And you're kind of what you're seeing now is just like everybody that couldn't walled garden ify themselves. Like look at the New York Times. It's a perfect example, right? I mean, like if you look at their revenue breakdown, like a significant proportion of their revenue comes from games now and it's all based on the subscription, right. They did a perfect job of transitioning away. And then when the revenue does come from ads, it's from their own proprietary ad tech. They built a really scaled subscription business with like a number of different value propositions. Everything that's left on the open web is just the stuff that got picked over and couldn't do that. It couldn't make that transition because they don't offer enough value to actually get people to even, maybe even just sign up. Right. And to get past even a free paywall or like a authentication gate. Right. Then what's the value of that? It's like the dollars have shifted over to the high value properties and like whatever's left is really just kind of like a, you know, just, it's just bargain hunting, right? It's like trying to, you know, find gems. And so like, I think I just accelerates a lot of that because you're not going to want to have your content just freely accessible in that way because it'll get ingested and not credited. And so, you know, you're going to see, I think a lot more people just erect these, even if it is just a free authentication gate, but probably a lot more people shifting into paid models because they work really well. And particularly if you're like a niche subject, like advertising is never really going to be that, you know, scalable for you. So you'd rather just do, you know, the substack. My sense is like, you know, we're sort of like past the open web era already. What exists is sort of in a state of like perpetual decline. And like, I think one of the things that people are sort of like mistakenly celebrating is this idea that like disrupting the sort of like Google monopoly around, you know, its ad tech business is actually going to hasten the demise. No, it's going to accelerate it. If Google has no skin in the game, they're going to shift all their resources into AI overviews. They have no reason to support the open web as such. Right. That to me is just, it's not a cause for celebration. It's actually like there was probably one stable piece of the foundation holding the open web up and it was the ability to monetize it through Google's properties. And if they lose interest in that, like, I just see no path forward.

Yeah, Mike, I mean, what, what do you tell your friends who are still like leading growth at retailers or at brands? Would you recommend investing in the website?

Yeah, I mean it's part of what makes growth fun is that the set of challenges changes every few months. And I think growth people tend to be very good at designing an experiment, roadmap to kind of validate the next thing, but I think could sometimes be maybe short to medium term oriented as a result. And so frankly, I think the stream of optimizing website has often hit diminishing returns by now anyway. I don't know that people are thinking long term enough and like, hey, is this thing just going to go away altogether? I think there's probably more excitement on what's to come than there is worry on kind of what's starting to degrade. And I think more of my growth friends are spending more time thinking about the exact topics we're talking about today. So how do I go and increase discoverability in these new channels where all the impressions are coming from? How do I go take advantage of the fact that there is some alpha back in being a savvy media buyer now? There maybe wasn't a handful of years ago, but I'm having fun doing just meta paid marketing still. How do I go and kind of think about mcp? And we might be a little bit early to this, but in a world where we have all these different agents that are transacting with my brand, making myself transactable to them. So I think those are more of the conversations right now. And I'm not skirting the question, Rishabh. I just hear, I hear the website come up like less and that's kind of the reality of it.

Totally, totally. I'm going to just like use one of the things you said to Taschen's last topic, which I'm really curious because Eric is, if nothing the best documentation of meta in history. So there's this thing that Mark is now starting to say over and over again, which is like, you tell us your goal and we'll just do everything and we already do it better than most humans. Right. And so there's this question of like, do we just end up in a world of agents and what does the role of the agency become when that is actually the way that the platforms behave and especially meta. Right. And so I think it started three earnings calls ago. If I'm getting. Eric can probably correct me if I'm getting this wrong. It started three earnings calls ago where he said for the first time you're just going to tell us your goal. And then recently I feel like he's been on multiple podcasts where he's repeating the same message over and over again. And so what does that mean in terms of what is the actual role of the agency in that world where you can just tell meta what your goal is and or what is the role of the media buyer where you can tell meta what you want? And that's basically it.

Yeah. So I think like the role of the media buyers is already changing fundamentally. I mean like that already is kind of the modus operandi, right? If you're, you know, overseeing scale budget on meta, you're trying to lean in to the greatest possible extent to asc, if you're E Comm or Advantage plus for apps, if you're selling app installs or buying app installs. And so like I think that has already kind of fundamentally changed and I think like that's better. It's objectively better. Like people complain about it. No, that's exactly what everyone wants, right? I don't think people, people don't like acknowledge the sort of like profundity of that shift in strategy. Right. Like that was brilliant and it was perfectly timed. It was a result in large part of att. So it was kind of forced. But I mean it demonstrably worked really well. I mean look at the stock price, look at the revenue, right? But the other thing is like look at the outcome. It's facilitating commerce to a sort of like a larger magnitude than it was before, right? This is what people want. So it's not like, oh, you're killing jobs. No, this is what everyone has been clamoring for. And so what does the role of the media buyer become? Well, it becomes managing that, right? But also it becomes thinking through like strategically of how all these things work in concert. Because UA or whatever you want to call it, advertising, media buying, that doesn't exist in isolation from the product, from how the user engages with the product. And like the sort of like the tighter those two things are when you consider that entire like end to end user journey, like the better it's all going to perform, right? Like it all flows through. These are not. You can't analyze these things in isolation. You can't build for them in isolation, you can't optimize them in isolation. These are one and the same. And so I think, you know, that media buyer becomes more of like a product analyst, they become more of an economist, they become More of someone who thinks about like, how can we reduce that consumer surplus? Right? And so that is the real value add of a lot of this stuff. It's not cost reduction. It's not, oh, I get 10 creatives now for, you know, 50 bucks instead of one. It's not, oh, you know, I'm creating a lot of assets for free. No, it's, I'm actually improving outcomes for my consumers, for my customers. And they're spending more money as a result because they're getting more value out of the product. And then I can spend more money on user acquisition. Like all of these things fit into a cycle and sort of like the more unified that is, the better it is for everyone. It's not zero sum.

Maybe I'll approach this topic from a different angle, which is that of the agency. And I think we're all seeing all these marketing workflows, from creative generation to media buying and measurement. They're all becoming agentized where sophisticated AI agents could handle 90% of the work that mid junior level marketers used to do. Does this replace the role of agencies? Does it help agencies operate at a much better margin? How do you think about the role of agencies in this world?

Yeah, it's interesting, right? So I invested in a company called Gigi and they're building the kind of AI control center, the AI nerve center for agencies. And I was kind of like, well, you know, the sort of like the thing you hear about agencies is like they have a lot of really young people that are working like very long hours for not very much money. And so like how, how much are you saving by shifting that over to like this kind of AI system? And the point was like, there's so much domain knowledge just sort of trapped inside agencies that can be unlocked within AI platform and then sort of like brought to bear on clients that like, yeah, there's some cost savings there, but there's actually just like more value to lift delivery, right? And so like, okay, that actually makes a lot of sense because if you think about like a lot of what you're paying for as an agency, I think a lot of it's just like FaceTime, that's not really value add. I always come back to this example, right? So like I was giving a talk at this event and it was like a brand event. And like I don't normally give talks at those. It's like kind of different crowd, you know, different circle. And so I was talking to some people and I was like, so, you know, is it true? Like I Hear all these stories all the time that you get brought to, like, the super bowl, or you're getting brought to the Beyonce concert, or you get the box at the. At the basketball game. And they're like, yeah, that's why we love our job. That's why we love doing what we do. It's because, like, we basically get, you know, once a month or maybe twice a month, we're invited to some event, and we get to sit in the box. We get to sit in the floor seats. And then someone was like, you know, at my agency, we throw a big Christmas party every year, and it's sponsored by the SaaS tool providers. Like, the Christmas party is paid for by these SaaS tool providers. And, like, they compete with each other to try to be, like, the main sponsor. But one year, like, there was one company that they really wanted it, but, like, they just. They were too late or something. And so they said, look, we'll throw you a bone. You can sponsor the brunch that will serve in the office the next day. And I was like, so you do business with these people because they sponsor your brunch? You're, like, hungover post Christmas party. Like, that's a reason to buy their product. And they're like, yeah, because, like, they got tacos. They had mimosa. And I was just like. It was like that scene in the big short where Steve Carell's character, Mark Baum, was like, why are they confessing? And then his. You know, his junior was like, they're not confessing. They're bragging.

Oh, my God. That's the funniest thing.

Speaking of which, Fermot just launched their.

Yeah, I was gonna say you don't like tacos enough.

Eric, we gotta get you. We gotta get you some nice tacos. That's one of the funniest stories I've heard. It's especially funny because I feel like in today's world of cutting through the noise, you know, this whole, like, what is the only. What is the way to get attention is, like. It's one of the few ways to get attention is in person, especially in SaaS.

So, Eric, I know we're up against time here. This has been a great conversation, as always. We got through, like, half of what we wanted to, but maybe we could, you know, sponsor your holiday party next time and get you to come back for another one. Thank you so much for doing this. We will chat again soon.

Yeah, nice to see you both. Thank you so much for having me.