AI Search Grader

Foreign. Let's talk about how to win in the new era of AI search with AI searchgrader. Jennifer was kind enough to present to. To share her speaker notes with me, but I am actually not going to use those. I'm gonna. Asia, you can. You can tell her if I did a terrible job later, I'm open to all the feedback, but this is actually a thing I care very passionately about. I have said on many, many threads internally, and I'm telling you all now, I feel like we are at a very crucial moment, not just for HubSpot, not just for technology, not just because of AI, but for the concept of inbound as a whole. And the way we connect with our customers, the way we meet their needs and build their relationships and grow our businesses is fundamentally shifting in the same way it shifted we. When Brian and Dharmesh first came on the scene with this idea of inbound marketing back in 2006, right? They had this crazy idea of, hey, this Google thing really seems to have big implications for how businesses are found and how they connect with customers. And you can make this inbound motion where people come to you and you do business with them. They built HubSpot on top of that idea, right? And now, almost 20 years later, everything is shifting again. AI is not just. Everybody loves to talk about generative AI. And, yeah, you can use AI to create emails, you can use AI to create website pages, but the thing that I think is a much bigger deal is you can use AI to find products and services, and that completely changes the game for what it means to get found online. And so I was delighted. I was so excited when HubSpot launched AI Searchgrader, because I had been saying in my own mind that this friends, you know, like my brother at a family reunion, as we're sitting in a campfire, like, I'm really worried that HubSpot's gonna get disrupted by someone who figures out what the inbound 2.0 looks like. I don't know, is HubSpot doing anything about this? And then, like, the next week, HubSpot launched AI Search Creator. And I was like, all right, HubSpot, you're always ahead of me. I was so glad to know I'm not the only one, like, wondering. So I'm not Jennifer Stone, but that is Asia Frost, and that's good news. So we're going to talk a little bit about setting the stage. AI's impact on search. We're going to talk about. Asia is going to give a demo of AI searchgrader and what that is. And then we're going to have a live Q and A. And I don't know how much Asia loves to hear people talk about her, but I'm actually a big Asia Frost fan. She's been a HubSpot doing SEO stuff for a long time. And HubSpot is like, we're pretty world class at SEO. We've been getting found on Google real good for a long time. A lot of that is Asia's work. She has not been blind to the changes that has happened. When we first launched AI Searchgrader, HubSpot's own scores were not fantastic and Asia has righted the ship. This is an incredible person to have access to here. I will hand it over to her as quickly as possible. Just because you're good friends here. I'm Kyle Jepson. AI has entered the chat. This goes without saying, AI is everywhere now. Everybody's talking about AI now. You can't look at your news feeds or be on social media. You can't talk to your friends, you can't talk to your boss without someone mentioning AI at some point. And the consumer search experience is evolving. Maybe I should have had these slides as I was giving my rant at the beginning because they rhyme pretty well with what I was saying. But people are using things like Perplexity and chatgpt and Claude to get answers to question. And Google is evolving with Gemini and those AI powered searches, snippets at the top of their, at the top of their responses. People expect this more conversational. I can ask a question. It can be served up an answer. I don't have to visit a website if I don't want to. And, and that is a pretty fundamental shift. I've been saying this for a while without any data, but now look, here is data from just last month, no, from I guess a year ago. AI chatbot for search usage is up 37% while traditional, traditional search engine usage is down 11%. And if you think about how big of a pie traditional search engine usage is, 11% is. That's a lot of searches that just disappeared, right? And this 37% growth in chatbots, I imagine that is continuing accelerating. This now 2025, this is the world we live in. The ground is shifting under our feet. And those of us who have been doing inbound for a while and have been very kind, confident in the best practices we've always known and loved, we should be paying attention and we should be trying some new things. Disruption in the digital search experience is Kind of the main headline for this session but also this is something I hope you all are thinking about again. I think it's really fun that you can use AI to write or proofread an email, but I don't think that's the biggest implication for your business right now. I think the far bigger implication is if you've been getting lots of leads from Google, you're getting 11% less now. Right. And if you are not getting any from these new chats, that is, that is, that should be a big concern for you because that is the future. So the big question, what now? And I had this question which is why I asked Jennifer and Asia to come present at the admins hug. I had the question, I didn't have the answers. What do we do? And there it's interesting. I don't know if you can see these. These are actually really good sites. Similar prompting provides different results. So if you, if you ask what is the best CRM, ChatGPT will give you one set of answers. Perplexity will give you a different set of answers. Gemini will give you a different set of answers. And so it almost world where we used to just solve for Google and now there are so many things to think about and they're all playing by their own rules. But the first step is understanding how you're currently doing. Right? If I asked you today, how is Gemini representing you? How is ChatGPT representing you in chat results? That can be a very hard thing to know but we have built AI searchgrader to answer that question and it's very exciting. At first it was just ChatGPT. It would do some ChatGPT queries and it would let you know how often you were mentioned and how positive or negative, like a sentiment analysis. And then a couple of months ago, I think right at the end of the year we introduced Perplexity, which I love. That's now my default search engine. I don't use Google anymore, I use Perplexity and so I'm really happy to see that here and more coming soon. We are working on Google Gemini, that's the next one and we hope to add more after that. So this is to give you visibility into how your company is doing right now in the same way that in 2006 one of the first things HubSpot ever launched was website grader. It just give you some data. How is your website performing now? You know where your areas of growth are. We've started in the same way now giving you visibility into these metrics that would Otherwise be very hard for you to. To understand. So with that, I'll pass it over to Asia, who actually knows stuff, and we'll teach you things I don't know.

Kyle, that was a pretty incredible intro for, you know, DIY in those slides. Hey, everyone, I am Asia. Kyle has teed me up really nicely. My formal title is Senior Director of Global growth here at HubSpot. As Kyle alluded to, I have been working on HubSpot's search strategy for about nine years now. So I have been in up around HubSpot's keyword and content marketing strategy for a very long time. And as Kyle alluded to, I. I think I got nervous probably a little earlier than he did, which makes sense. You would hope I would get nervous first when I realized, hey, we have a really good understanding of how we're ranking on Google, how we're driving traffic from Google, but not only do I have no visibility into these different LLMs, or when I try to get that visibility, I'm spending about 15 hours a week. It was an inordinate amount of time going to each LLM, typing in the queries that I knew our prospects were currently typing into Google and would one day be going to platforms like Perplexity and chatgpt to find answers to and then manually synthesizing the results in a spreadsheet, which is actually kind of ironic because AI across the board generally makes analysis and fact finding easier. This was making analysis and fact finding a lot more laborious, which is where the idea for AI searchgrader came into being, as these things usually do on a long walk. And I thought, oh, this is a problem for me. This is absolutely a problem for our customers. I am now going to. Oh, I don't have screen sharing. I did have it and now it's gone.

I'm still sharing. Sorry.

No worries.

Okay. Yeah. Okay. There you go.

No, you're just keeping me on my toes. Between that and the chat, I feel alive.

Love it.

Can everyone see my screen? I actually can't see anyone in the screen. Yeah. So give me variable cues. Thank you.

It is. It's just black right now.

Yeah. Oh, let's see. Let me stop sharing and try again.

Yeah, I think I turn it off and back on again. Is.

Yeah, that's usually work. It always works in the demo and then.

Yeah, first shot out of the gate before we started officially, it worked fine.

How's that? Same thing.

Wait, there we go. You can see it now. It just needs. Yeah, all right.

Just making us work for it. Cool. So I'm gonna start by looking up my parents shutter company. My parents have a small shutter company in California. They make a hundred percent wooden shutters custom made in the states. So manufacturing company. And then I'm going to click grade my brand and I've already looked up Elizabeth Shutters. It's a company that I monitor the sentiment and share voice for an AI search results very often. But if my results weren't cached, what the Micro app, another name for tool. Sorry. What the tool would have just done is come up with all the different quiz queries that current and future customers are likely using to research wooden shutter companies as well as Elizabeth Shutters itself in LLMs and then said okay for those queries. How are we doing? In very simplistic terms. So we get a total score out of 145is. It's fine, it's not, it's not terrible. But there's certainly work to be done. And each score out of 100 has two components. So we'll walk through what those components mean. I'm also going to show you the difference between OpenAI and perplexity scores. Then I'll talk through what my parents could do about this score, what they are doing about this score and then I'll open up for questions. We can do a few more demos if folks want to see other companies. Does that sound good? I feel so lost without any visual or.

Yeah, no, that is the disorienting part of a web based webinar. Yes, we are here already. Lock a thumbs up in the chat.

Great. All right, the largest of the score is brand sentiment. So we use sentiment analysis to look at the language that LLMs are responding around your brand. And then each response is categorized as positive, neutral or negative. If the response is largely complementary, it has a negative language. But like the whole shebang is largely positive, it'll be positive or the reverse. If it's in the middle, it'll be neutral. What you obviously want is as much positive as possible and as little negative as possible. But it's really helpful to understand, okay, what about the brand is coming up with really great. What it. What is the LLM calling out as something that's really strong about the brand and what is the LLM calling out as negative? Because those are things that you want to address not only in your response plan for optimizing LLMs, but they're probably things that you need to focus on with your customers as well. So strengths for Elizabeth, quality, craftsmanship, use of 100% American hardwood a wide range of customization options and eco friendly practices. These are all really good to see because I can tell you these are things that my parents have very much focused on in their branding and their competitive positioning. However, we know that 6% of the responses are negative and those negative responses concentrate on pricing concerns. Elizabeth Shutters is definitely on the higher end compared to other providers. That's kind of the cost of making everything in the states and using hiring materials. Customer service feedback is mixed. So there are some things that my parents will definitely want to pay attention to there because these are things that if customers are finding them from LLMs, it's going to be top of mind. And then finally some installation process issues. Okay, then we get to Share Voice. And Share a voice is really, really interesting. Share Voice looks at unprompted category mentions and says how often does a brand show up in those? That was pretty jargony, so I'll try to say it in a simpler way. If I give ChatGPT 10 different questions about wood shutters in California, how many times is Elizabeth Shutters going to come up in the responses? According to AI search creator, zero times. So we can see here who is coming out. We have Hunter Douglas. That's a really big player in the shutter world. I don't know if everyone's just learning way more about shutters than they anticipated, but I can tell you this is my childhood. We have Danwer, we have the shutter company Sunland. You can see that the Chat GPT is still hasn't yet conflated Norman Shutters and Norman usa. Same with Danmer. So that can also be really interesting to note if your brand is not being conflated because that suggests that you need to help the model understand your holistic entity. That was also pretty jargony similar to making sure that Google understands that Danmer Custom Shutters and Danmer are the same company. You want the LLM to be able to do that too. And because LLMs are more nascent, their understanding of what a company is or isn't tends to be less mature too. So we can talk about how you can address that later on, but okay, Elizabeth Shutters is not on OpenAI's radar for custom with Shutters in California, who is, as we kind of called out, looking at the graph, Hunter Douglas is in the lead. Then we have a few other players. And what I really love, I mean I say this like I wasn't part of this. One thing I think we did really well here was calling out specifically how your competitors are being cited in the search results, so that you can either try to imitate what's going really well or find your own niche positioning. Okay, and then let me show you the score that Elizabeth Shutters gets for Perplexity. So what is obvious is that the Perplexity score is way higher. And having run thousands of these queries at this point, I can tell you that this is pretty common. And I think it's because Perplexity queries in real time, whereas OpenAI has a knowledge cutoff. Because Perplexity is updating in real time and has a wider corpus of data to pull from, it tends to have a better view of how your brand is doing, particularly if you've been doing a lot of work to improve these scores. So that's something to pay attention to as you look at the results. And I don't want to miss the key point that you can perform very differently in one LLM versus the other. If you've been in search for a long time, you probably know that performance on Google is pretty analogous to performance on Bing or there are some differences globally for Yandex, say, or Yahoo Search in Japan. But all of these search engines at this point have pretty similar algorithms and we can see that that's very different for LLMs at this particular stage. So, same thing. We have brand sentiment, share voice, but brand sentiment is a lot higher. We actually have no negative mentions. And then share voice, we have much higher share voice. And again, that's likely because Perplexity is pulling data in real time. Okay, I am going to stop sharing for now. I can always start sharing again if we want to do another demo. But I want to talk through some of these specific strategies. Oh, I already see some questions. Okay.

Go ahead. I would love to hear your specific strategies. They may answer some of the questions and then.

Yeah, yeah, yeah, no, I. Those are good questions. I want to get back to them. I want to talk about some of the specific strategies that HubSpot has used to improve sentiment and share voice in LLM responses. The first is around understanding where LLMs are pulling data from and making sure that we are appearing as positively and prominently in those places as possible. So one of the key places for training data is user generated content or forums. That means Reddit, that means Quora, that means Stack Exchange. And depending on what your company or brand does and where your prospects or customers hang out, those are going to be different forums. But we identified those key places for HubSpot current and prospective customers, and we are spending a lot of time there Understanding what the conversation is, getting involved in those conversations, starting new conversations with the goal of stimulating positive conversation about HubSpot. Now you have to be very careful, especially on places like Reddit that are pretty allergic to manipulated or inauthentic results. So I would say this is a very important, important tactic in the world of LLM optimization, but it is absolutely not one that you should just go whole hog on. Do not go back to your desk and start joining or starting Reddit or Quora threads about your company. You have to understand what people are talking about, why they're talking about it, whether they're talking about your company already or not, and adding a lot of value before you ever get to and XYZ is really great for abc, but if you can create authentic genuine conversation, we know and we very clearly see the results of getting those positive mentions in user generated content. A more straightforward one for a lot of folks. Sorry, I just got pop up. I feel like there's never a dull moment in these, all the pop ups. Another more straightforward one, which is a really interesting one, is publishing a book. Now this does not have to be a massive tome or anything of the encyclopedic level. It could be an ebook that you've previously shared to your website, but publishing it as a book can get, it can get the content introduced to Google Scholar and other kind of course corpuses of Data that again LLMs are pulling information from. So that can be a really good and relatively simple way compared to some of these other more indirect methods of getting more visibility and getting more positive association with your brand in these training sets.

I just want to pop in here and say the thing I really love about that suggestion is I attended a webinar where Asia and Jennifer presented very similar content just, I don't know, a month, six weeks ago. That's a new one. These really are evolving really quickly. But yeah, that's, that's really cool. So like if I have an ebook, just go publish it on Amazon or something is what you would recommend?

Yeah, I mean make sure it's good quality. I would say for all of this, make sure it's good quality. So also don't just use ChatGPT to write you an ebook and then publish it. That, that would not be the way that I would approach it. But yeah, publish to publish three and to your point Kyle, it's a very quickly evolving space. Another strategy that is a very new strategy is having, well, I'm going to say it and then I'll explain it. Robots TXT for LLMs. So if you're a technical SEO or you've dabbled in technical SEO, then you are very familiar with this concept. But if you're not a robot, txt file folder lives on your website and is a directive for search engines telling them how to crawl or access or not crawl or don't access your content. So you could say, hey, crawl these pages. Don't crawl all of these pages. Now you can have an LLM txt so you can say to AI bots and hey, look at this content or don't look at this content. You could say I'm okay with perplexity, but I'm not okay with ChatGPT. You could say I'm not okay with any of them. If you don't want your content to appear at all, that's not something that I would recommend. I would recommend figuring out what it is really important for LLMs to understand about your brand, your website, your customers, et cetera, and making sure that they can explain explicitly. Crawl that content again so that you have more visibility in the results. But there may be things that you really don't want to show up in search results. And so those are the things that you would say, don't crawl this. So that's a, that's a new tactic. We were not talking about LLM Txt when Jennifer and I co presented in December. Yeah, I could go on and on and thank you chat for appreciating the passion. And one of the holistic concepts that I definitely want to communicate is brand mentions are the new backlinks. And what I think is kind of funny about LLM optimization in general is that it's what was old is new again. I mean publishing books, we published an inbound marketing book many years ago as a PR move, as a way to spread the good word of inbound. Now it's an LLM optimization technique, making sure that people are talking about your brand in really good ways wherever your current and future customers will hang out. That is not a new marketing concept by any means. That is community building. That is community management, Kyle. It's what you excel at. So I don't want anyone to feel scared. I think you should feel excited because a lot of this is new applications of time tested techniques. Okay. Anything I should clarify, Kyle, before we open it up for questions, one thing.

I'd like to ask, and again, because all these best practices are evolving so quickly, it's okay if you don't have a ready answer for this.

Sure.

But I imagine there's A pretty big difference between how an organization like HubSpot thinks about and can execute against these strategies. And like your parents shutter company. Right. Like if I think about like HubSpot, we have like a Wikipedia page. We are mentioned all over Stack Exchange and Reddit and stuff we have. There is an R HubSpot right on. On Reddit there probably is not an R elizabethshutters. So for, for the people here in the chat who, who feel closer to that small business side of things, recognizing that they use HubSpot probably since they're here.

Yeah.

What is there, is there someplace. What would you say is the, the best place to start? What's the first like AI searchgrader. Obviously figure out what you're doing. But if things are not as good as you would like them to be, where, where would you recommend they. They turn to first?

Yeah, I specifically use my parents shutter Factory and not HubSpot as an example because I do think that the practices are pretty applicable across the board. It is the scope that changes. HubSpot has the resourcing the people to do the practices that I talked about on a very large scale. But my parents cannot do that. But my dad certainly has the time and should be going into Reddit on a monthly basis seeing if people are asking for window covering recommendations in California coverings and offering his expertise and disclaiming, hey, I run Elizabeth Shutters. We do custom wood shutters. If you have any more questions, let me know.

Yeah, they should be looking at their.

Reviews on Yelp and making sure that they're responding to those and encouraging customers to leave new ones and Angie's List. And so I actually, I think everything that I said, even the book, because I think that's pretty straightforward too are things that small, medium and large companies can do.

That's really interesting. Yeah. And, And I mean HubSpot, like I said, there is like an R HubSpot, there's a HubSpot Wikipedia page. But for everyone here, their company is part of some industry and there is likely an R slash, whatever that industry is. Or, or for something like shutters like you know, homeowner threads and just like interior design forums and those sorts of things, there are, there are categories where you belong and you should be playing.

Yeah. And maybe it's not Reddit. I can see some people saying Reddit is not relevant to all businesses and that's completely accurate. Maybe it is Stack Exchange, maybe it's Quora, maybe it's some form I don't know of because it is so niche to as long as it is crawlable. So as long as it's not in Slack or somehow gated so that LLMs can't access it, it is very likely that they are. Because the other big. It's not a secret, but it's something very good to keep top of mind as you navigate this world. LLMs need fresh data more than anything else. You have to keep feeding an LLM fresh data for it to keep evolving. And so what all these companies, Claude and Google and OpenAI are looking for more than anything is okay. Where do we get good stuff to give the models? If it's relevant, if it's user generated, which tends to be more authentic and not AI generated, it's probably getting swept into a model.

It's interesting. It occurs to me for so long with SEO and like website grader and stuff, we've been talking about your on site content and performance, right? This is all off site stuff. This is. You got to go out in the world and figure out where conversations either are happening about your company or should be happening about your company and jump on in.

Exactly, yeah. And don't forget about your website whatsoever. I mean your website is probably your what we would hope your website is getting crawled and ingested as well. That's why, you know, you set up an LLM txt. Another tip now that I say this is once you understand what those strengths and weaknesses are, can you create content speaking to them on your website? Those strengths and can you address those concerns? Basically every touch point that you have you should be using strategically.

Yeah, yeah, yeah. And that's that. That part of it is just still good old fashioned content marketing. Right? What are the questions people have about you? What are the concerns they have? How can you resolve those? You mentioned LLM txt again just in passing there in the chat, it scrolled away. I don't know where it is. Someone is asking if there's an LLM TXT template that could be used. Do you have a resource you can recommend for people, how people can understand how to structure that instead?

Yeah, there are a bunch of free tools that have popped up. I don't remember any of them off the top of my head, but if you search LM TXT Maker or Creator generator, you know all the acronyms, you should be able to find a few options. And I can also do a little bit of digging after the call to see if there were any in particular that I would recommend.

Yeah, looks like Deanna found one and dropped one in already. Cool. So I think at this Point, we will transition over to the actual Q and A. I'll stop hogging the question asking roll here. So if you all want to hop over there. Looks like we got about 20 questions right now. Everybody just skim through them real quick. If there's one that you really want us to get to, be sure to upvote it. If the question you're hoping is going to be asked isn't in there, go ahead and add it and I'll give you just, I don't know, a few seconds to. I will fill up my water bottle and then, and then we'll do it.

Okay, I will browse them. Why while you do that? Yeah, I don't, I don't claim to know anything about that one, but it was one of the first ones that came up quickly. Sorry, which one? The Fire crawl. Oh, yes, the LM Txt generator.

Okay, that was fast, guys. The kitchen is just right next door, but it is less fast than me being like, I'm going to wait 30 seconds and then seven seconds later being like, yeah, that's probably 30. Let's, let's hop in now.

It's never 30.

Never. I've never once made it there. Okay, I'm going to sort the Q and A by Most voted currently with 19 votes, Angelica asks, what strategies and techniques can we use to improve visibility and in AI search results? I'm noticing this question came in at 11:14, so that is mostly what we've been talking about. I'm going to consider that one answered already, but if anyone has follow up questions, if you have specific questions about techniques or whatever, go ahead and add those. We will. We'll get through as many of these as we can. Daniel asks, when do you plan to integrate the SEO analysis from HubSpot into the AI search creator, if at all.

It's a good question. I cannot comment on our roadmap, but I will say that I could see those things being useful if combined.

Love it. I, I will say, I just. HubSpot has been doing a lot of these little micro apps lately. I think they're really fun and I feel like it's a great way for us to test new functionalities and stuff. Be cool to see them start merging and stuff. I, I don't, I don't, I, I don't actually know anything about the roadmap. It's not that I can't comment on it. I haven't seen it.

I will say we have a precedent, so we have integrated a few micro apps for folks who aren't familiar. That's a very HubSpot specific term, but we call that tools that sit outside the core product that usually accomplish one or two things really well and they usually exist as an in ramp into the product. So you can understand the value of something HubSpot does before you actually sign up or buy. And last year Campaign Assistant, which is a AI generated assistant helper, helps you build assets for marketing campaigns. That functionality was integrated into the product. So there's a. There, there's some precedent.

It can be done.

It can be done. Exactly.

All right. Riley says this is a really zoomed out question so feel free to ignore. So I'm not going to ignore because I'm a youngest child and I don't ignore things. But how are we speculating that paid might impact this world? I might be jaded, but the almighty dollar usually opens all doors. It's probably a matter of time until companies want to start throwing money at these engines similar to how they threw money at Google for better performance in their search results. Do we have any ideas slash nightmares for how this world will handle attempted monetization?

Yeah, this is a really good question. I'm so glad you asked Riley. I think we are starting to see this happen. So Perplexity is the only LLM true platform, I'm not counting Google, that has introduced paid ads within its responses. And I have no data here, but I would be surprised if the companies that are paying Perplexity for those ads did not get some sort of advantage in responses. Again, I can't say for certain. That would just be my gut. Google has never publicly said that folks that pay for ads get more responses in organic. So it's not something that we're like, okay, this is how it worked for Google. This is how to work for these lms. That's just my hunch. And that would be my hunch for the other LLMs as well. But it also opens up new opportunities for paid. So if your audience is migrating off of Google onto these LLMs, if you're spending money on paid, then you can start to migrate that ad spend. I don't think it's something to be afraid of necessarily. It's just something that you'll want to consider as your customers evolve and then your strategy needs to too.

And I think just in general, the markets and the investors are all really interested how these companies are going to monetize. So keep watching the headlines. I'm sure there will be developments on the monetization front before too long. Jeanette asks my CEO wrote a book years ago. Should we publish independent chapters as blog posts or how do the LLMs best find the book?

If you think the blog posts are going to be helpful to your audience, then I would publish them. I would not publish them specifically to appear in LLMs. If the book was published years ago, I would check to see if it's in any academic or book databases like Google Scholar or the Kindle kind of open library.

Great. Is there probably the future plan to integrate AI searchgrader analytics results slash optimization suggestions eg for website Pages directly into HubSpot environment as like it works for integrated SEO features. So far I think we, I think we touched on that. I don't know.

Yeah, same answer on the roadmap. Thank you for asking.

Well, we like the way you're thinking everyone. How can you improve the score on AI searchgrader? To what extent can you improve this score? What strategies would you recommend? This is another one that was asked pretty early on 1122. So you've touched on a lot of this, I guess perhaps the way I would reframe it is I know you know, from the earliest days of HubSpot, people come on board and we say inbound marketing is not fast. Right? You work hard and, and it takes a few months. And if, if you don't see results week one, don't worry, you're not supposed to see results in week one. Is, is this space different in that regard? Do the, do the scores move faster or, or what are. Have, have you noticed trends there?

Yeah, really, really good question. I think it moves slower depending on the LLM. So I saw there was some ch slack about okay, is AISG providing answers pulled from Search GPT or ChatGPT? Search GPT provides answers in real time. It queries the web like Google does. Chat GPT does not query the web. And I can't remember when the latest knowledge cutoff was. I want to say November 2024, maybe earlier. So what that basically means is that the results from OpenAI or not OpenAI, sorry, chatgpt about your brand are going to be static until the next knowledge update. You could put in a ton of work in between now and whenever the next update is and not see any changes and then you could see big increases. So it's not going to be something that you see overnight. Now for perplexity where there are real time data and those things update very quickly, you could see results overnight. Keep that in mind when you're looking at your various scores.

Got it. So this next question, what is the plan slash timeline to add more AI engines. Now this is, this is my bad. If I had actually read Jennifer's speaker notes, you would have gotten more detail. So when I said Gemini is coming closer, her speaker notes say in the next few weeks we'll release Gemini which powers Google's AI overviews and we're continuing to add LLMs and more functionality to AI Searchgrader. Exciting, right? So sounds like Gemini is very soon. And then, I mean she doesn't mention that sort of exhausts my knowledge of. Of LLMs you might use to search for businesses.

We don't have support for Claude right now, which is Anthropics LLM. But Gemini is very exciting because Gemini is engine that powers AIOS. Now you have ChatGPT, which is kind of the front runner model. You have Perplexity which offers real time data and then you have Gemini which powers aios. It's a pretty sweet combo in my humble opinion.

And. And Asia cannot comment on roadmaps and I don't know roadmaps, but Jennifer speaker notes say in the next few weeks. So that, that sounds pretty.

No, I can comment on that. I can confirm. Yeah, in the next few weeks, maybe sooner.

Okay.

Hi out. Yeah, stay tuned.

Keep an eye on AI search creator because I'm such a fan. If you follow me on LinkedIn I will certainly be announcing when Gemini reaches it because I. That is a big one. A lot of people are sticking with Google and so that matters. Are there limitations on the location of companies or specific markets where the search grader doesn't work properly? For example, for a small business on the financial and baking sector in Sao Paulo, Brazil? I'm guessing that's not hypothetical. No.

Right. Just randomly asking for a friend. No, there are no limitations on location. I will say that at. Let me back up. The purpose of AI searchgrader is to give you the results that your current and future customers are going to see to to be a mirror of what they are experiencing. So if you are a small wooden shutters manufacturing company that has no presence outside of the town of Redlands, California, if you put in US for your location, your share of voice is going to be very low because you're just not going to register on that scale. You would want to put in Redlands or Southern California or Inland Empire something that is more representative of the location that your customers or future current would be entering when they use aisg. So that's something to keep in mind. And I saw some other questions kind of coming up around both location and company type. And I would say try to put yourself in your customer's mindset when you fill out those fields because that's going to give you the most representative data.

I love that. And that's the thing that for me, just as a, not an SEO expert, just a guy who uses the Internet sometimes resonates. Right. I have definitely gone to my phone and been like, what's the best pizza in Boston? Right. And, and there are places that show up that would probably not show up if I asked what are the best pizza places in the US or in the world. Right. So curious to understand why only company name and not URL is part of the input fields for AI for the AI grader.

Yeah. Fun fact. We originally had company URL as one of the inputs. It was, it added a lot of time because we were, we created a separate model and kind of a separate step in the process of going to your website and crawling your website. And what we found was that the quality of their responses barely changed. So we eliminated it so that we would have a faster, more efficient tool and we could get you to those insights more quickly. One thing that has been very interesting and cool about working on AI searchgrader is that the models evolve so quickly that something that didn't necessarily work for us six, four, even two months ago can be very effective for us now. I tell my team, never say never. That doesn't mean that we'll never go back to adding website into the inputs and potentially providing website specific insights and functionality. It just means that when we are making that call at the time of launch, it didn't make sense for the user.

That's cool insight, Robin. Thanks for asking that question. I did not know it. Uh, Pedro asks, does video count for lmo?

It does, yes. So LMO for the person who asked for an AI acronym translator slash. Anyone else who's not familiar means language Model Optimization Artist, also known as generative AI optimization, sometimes called AI optimization. There's a whole host of names, but it basically means how are you trying to influence LLMs for better results? And, and yes, video is another place that LLMs are pulling information from. Again, it tends to be user generated content. It tends to be pretty good quality from a data perspective. And if you're not doing video, we're also seeing customers increasingly migrate to video because it is one of the areas where there is less gen content in general. So folks tend to feel like they're really hearing from a human, which is something they're starting to value more and more. Two good reasons to think about your video strategy in 2025 if you're not already.

Cool when testing the search grader, I get a score of zero and it says an error occurred. Refresh results underneath both OpenAI and perplexity. What can we do in this instance to troubleshoot and get our grade?

It happens because it's a newer tool. So what you can do is email me the inputs that you used. My email is. It's okay to say my email, right Kyle?

Sure, yeah. They would love it.

Cool. Don't make me regret it. My email is afrostubspot.com and I would love to troubleshoot it with the team.

Cool. Thanks for that.

And if you want like a quick response, you could also try going back to the homepage and entering everything again. If you're repeatedly getting it, then there's something going on under the hood.

Got it. So Google, indeed, Glassdoor, BBB and other similar platform reviews can affect your search grade. Right.

We believe that those platforms can impact your perception and share voice in LLMs. Which means that yes, they can indirectly influence the score that you get on AI searchgrader. But as a reminder, the AI searchgrader score is a proxy for what we believe your customers are seeing. So you want to be thinking about, okay, what are the sources of data that LLMs are pulling for and how is that going to influence the responses my customers get about me and my competitors. Yeah, rather than getting like, you know, myopic about the score.

Yeah, yeah, that makes sense. What if we get a score of zero? What does it mean and what can I do about it? So it sounds like that's happening to a few people. What publishing channels and platforms should be used for ebooks to help with AI search?

I wouldn't over engineer this. I think publishing a book to an open source database is the way to go. So you don't need to use a specific publisher, you don't need to use a specific channel. You just want to make sure it's an open source database.

Stacey asks, are other LLMs going to be added to this? Yes, Gemini is coming soon. Others as needed. How long does the brand need to exist before we expect measurable scores here?

That is going to vary quite a bit. As you probably expected when you asked that question. It's definitely one of those it depends questions. The first biggest factor is how narrow or broad are your inputs. So if you want to be best pizza in the world, that could be a very long time, if ever. If you want to be best pizza in Stonehampton, Massachusetts. Then that could be six months, that could be a year, could be quicker, depending on how quickly you make progress with brand mentions with a lot of the strategies that we've talked about. So I would say it's the scope is going to make a big difference and then how quickly you implement those strategies and how effective you want those are going to make a big difference too. So it's a big fat. It depends. I'm sorry.

That makes sense though. And that's helpful context. Sabrina also got a zero, so just so you know.

Yeah. Anyone. Anyone who's having like repeated trouble with the tool, then definitely please email me and we'll get you all squared away.

And that's not just Asia being nice. That's going to make the tool better, which is something we have a vested interest in. Yeah, Asia is nice though. Can you ask the LLMs what sources they access the most for information on your brand?

That would be a really interesting prompt to go directly to ChatGPT or perplexity and ask Perplexity. You can certainly see the sources and that's something we're thinking about trying to integrate into the next version of the tool so that you don't have to go off tool to get those insights. But yeah, you could certainly ask ChatGPT. Now, whether or not you will get an answer is it's not. It's not going to be reliable and it's certainly not going to be holistic because the LLMs have a vested interest in kind of protecting where they're pulling data from. But I'm sure you could get interesting insights.

Yeah. If we identify our company is doing good on one platform but missing from another, what should our next step B? And then in parentheses searched best software systems in a particular category and our company didn't appear under ChatGPT but was number one on perplexity.

Interesting. Yeah. That aligns with what I had mentioned around Perplexity, usually giving more positive results. I would develop a specific plan for ChatGPT and execute on that, which congrats. Now you don't have to fight a war on two fronts. You've only got one front to focus on. And the other thing you can do if you are really looking to dominate is say, okay, that was one category. How do we do across all the categories that we care about? Maybe you only care about that one category, but most companies have a few.

When we see leads coming in with ChatGPT as a referrer, is there any way to figure out what they were searching for and how.

Excellent Question.

That's a good SEO question.

Yeah, there is no Google search console for LLM data right now and in fact LLM prompt data is pretty protected by the platforms. So no. You can however infer what they came in for by where they landed. So which page to ChatGPT refer them to and and then again the insights from AI searchgrader. If I see that we are getting the Most traffic from ChatGPT to HubSpot's free CRM page and we have good visibility for free CRM in our target markets, well, I can probably connect the dots that people are asking about free CRMs and getting directed to the free CRM page.

Okay, we're getting pretty specific with this next question. Is there any issue with using new Reddit company accounts to interact with users? Or is this something where existing accounts work better for any sense of authenticity? Additionally, how impactful are things like Google reviews for company mentions?

Yeah, I am almost across the board existing accounts are going to be better. Again, I just want to say this every time I talk about Reddit and particular proceed with caution. If you create a new company account and you go into a bunch of Reddit threads and just say nice things about your company, you will get ejected and quickly. If you have a long this is best case scenario, you have a long existing account, you've been participating in these communities for many years, you have a good reputation, you you've given a lot of value and you have a very natural way to disclose that you are working for this company and talk about why it might be helpful for the person or people that are already engaging in that conversation. Now I recognize that the number of marketers who knew that this was coming or lucked into this situation, it's probably a minority of us. So with that as your end goal, figure out, okay, how can I go from where I am now to that goal? Maybe that's someone at your company already has an account and they can become your Reddit advocate. Maybe you you just need to bite the bullet and start an account but focus on building up that reputation, building up that community presence, giving value first before you ever get to and let me tell you about Elizabeth Shutters.

Amazing. We are basically at time here. We have a long tail of questions we're not going to get to, which is normal for the HubSpot admin hug. Thank you all for being here and for asking questions. Asia, thank you so much for being here and answering questions. Any last things you want to share before we shut it down?

Anytime I talk about AI, then I think people assume I am nervous and maybe even frustrated with how Google search has changed. Let me be very explicit. The changes are scary and they can be frustrating. But my overarching emotion more than anything else is excitement. Kyle, you said it perfectly. We haven't seen a shift of this type since Google came onto the scene. And that was when Brian and Dharmesh developed the concept of inbound marketing, which probably has really impacted the lives of everyone on this call. And I am so excited, excited for another moment like that. How cool that we get to be a part of it and we get to develop this next wave of techniques and concepts. So I hope that leaving this call y' all feel excited too.

Love that. Thanks for ending us on an optimistic note. Hope you all are feeling optimistic now. And that's. That's the end of today's show. Thank you all so much for joining us in another session of the HubSpot admins hug. Hope to see many of you back next week and go enjoy your Tuesday. Bye everyone.

Have a good one.