AI bots now visit websites as a distinct class of visitor that affects business outcomes. The speakers discuss lm.txt files as a potential future parallel to robots.txt for guiding bot behavior, and share best practices for designing sites that work well with AI retrieval agents.
How can we optimize that experience so that we have a better outcome for our business? And so it's kind of an interesting time where that is the case where people are noticing that like AI bots are a super important new visitor to your site because those AI bots are doing work on behalf of a human that's prompting it.
Is there an emerging standard around this in terms of like a, like a parallel to like robots.txt?
Yeah. So there, a lot of people have published articles around You know, lm.txt and actually like those, at least right now, those, those lm.txt files don't do anything other than they're just like a regular webpage on your site. So like, yeah, the LM is going to crawl it, but they're not going to like take it as direction like they would a robots.txt. So maybe it's a good hedge or something to include for the future, but I don't think it's having much noticeable impact than just like any other page would. I think there's a lot of best practices around like, okay, an AI retrieval agent.
Roll Claude out from one laptop to the whole GTM org | Sam Gong & Zach Vidibor
