How We Use AI In Our Marketing As A 35 Billion Dollar Company

On today's show, we're doing part two of AI Experiments in the Wild. I'm joined again by Emmy Jonasan, who is VP of marketing here at HubSpot. I'm your co host, Kip Bodnar, CMO at HubSpot. And we're walking through two areas where we've been experimenting in AI today. One is with AI driven chat. For us particularly, it's on your website, but you could be in your product. And we're going to talk about some new work we're about to roll out for AI on your website. Emmy, welcome back to the show. Thanks for. Thanks for joining us for part two.

Kip, thanks so much for having me back. I'm excited for part two and to get to more of these use cases that we teased a little bit in part one.

Yeah. So if you haven't watched part one, you can go back on YouTube or RSS and do that. We broke down kind of the use cases. We prioritized how we got there. We ran through the whole kind of preamble, and today we're going to talk about kind of the nitty gritty of two use cases that we have found a ton of value in and are really excited about and kind of share what we've learned with all of you today. We want to make AI practical and real, not some, like, philosophical thing that you could potentially be doing with that. Emmy, the first thing we want to talk about is chat. And I always. I always joke with people that, like, one of my dumbest moments in HubSpot history was when, like, I realized people called us to buy our software. You know, like, like, I was like, wait, they don't just want to, like, fill out a form and email with people. They actually want to call and talk to somebody. It turns out people want to talk to people in real time. Yeah. And so that at that time, we set up this team that we call the ISC team internally. And they did two things. They took phone calls from people who wanted to chat with us. And it was actually one person at first. It was a guy named Miles at first. And then they also took chat on our website. And could you maybe explain to people, like, how valuable a channel chat for our website is, both from a support perspective and a sales perspective, and then kind of start walking through what we've been working on.

It's an extremely valuable channel. Exactly. From both of those perspectives, I'll. I'll talk about kind of from a value perspective first, and then I'll transition into support. But in the last episode we talked about these kind of orbits of intent when we think about acquisition and demand volume. And you can imagine that if someone is coming to our website and talking to somebody over chat about our products, asking questions about our products, that that indicates a pretty strong level intent that that person may be interested in trying or buying a product. And so that that team that you, that you referenced is a real revenue generating team for HubSpot because they are talking to some of the most intentful demand that, that we are generating. And then from a support side of things, similar, you know, a lot of people are coming to our dot com experience, especially our knowledge base page. A lot of those people are customers and they're coming to ask questions around how can I do X, Y and Z in HubSpot? And the chat really gives a valuable tool to allow them to have a forum to ask those questions and get answers to those questions.

Yes. And one of the things you and I kind of talked about in prep for this is that a reminder to everybody is that one thing that AI is really good at is unstructured data and making sense out of unstructured data. And chats are like the perfect example of that. Right. It's just like somebody has this free text question in their mind and they're trying to chat it and get an answer to it. And as we've seen with tools like ChatGPT, that is the perfect AI use case. And so I think this was far and away our number one thing that we were going to go out and do with AI when we first started experimenting. We've been, it's the thing we've been working on the longest, maybe give everybody the setup. Like where did we choose to start working on what we do, how to go? Like just give us all the rundown.

And I'll definitely plus one LLMs. This is really where AI shines in. Being able to make sense out of unstructured data, understand that the intent of that data, what people are asking and then being able to return a really accurate result. And you really can't get more unstructured than a human being just typing in a bunch of free form text into an open text field. And so when we think about we are to start with chat similar to where we had a framework to decide which AI use cases we wanted to start with, how we wanted to stack, rank and prioritize them. We really needed to think about which page we wanted to start testing AI powered chat on. And that's because every single page across HubSpot is attracting a Persona with a different level of intent who is looking to do a different thing. You know, they might be looking to understand our pricing and packaging better. On our pricing page, for example, they may be looking to understand whether HubSpot can do X, Y and Z better by visiting our knowledge base. Or they may be looking to understand, okay, does the HubSpot Academy have a course that could help me learn how to do this very specific thing? So we had to be really intentional with where we wanted to start testing. And the two things we really thought about when we, when we decided which page we wanted to start with was one, we wanted a page that had a high volume of traffic coming to it because that would help us test, gather data and iterate very quickly. But we also wanted to test on a page that was lower risk because this was a very new motion for us. And of course, you know, if you're somebody who has experience working with AI training models, there's things that can happen as the model or as the use case is first released into the wild.

Okay, I want to pause here for a second because I think this is a really important lesson for everyone. We, we talked a couple minutes ago about the value of AI is that it is great at handling the unstructured data and free, free text data, free form data. Right. But when you're starting and you're just getting started, the things that I think we've learned and you just kind of outlined, but I want to emphasize is that you, if you can start where the data is, the, the unstructured data is as structured as possible. So for example, with like knowledge based, very product related questions, there's kind of a, a finite scope of things that people ask about a product. Right. It's not like everything under the sun. They're kind of a limited set of questions that are all similar types of questions.

Yes.

And then the second thing you said is like, oh, in having the most structured unstructured data to start working on, it's also like when you have the most information to match with that unstructured data, you're more likely to be able to create a really great result. And so if you have a big knowledge base repository of a lot of articles and that's a great place to start because you have a great base of knowledge to train the model on and for the model to match with those questions. And then the third thing you said that I think is super important is like enough people have to be going there that you get the feedback loop and you can and you can iterate and train that question.

Yes, that's right. And I think you summarize that really well, especially around when I say low risk and you said unstructured structured data, that that's exactly what we mean. Like we have, we have really good knowledge based articles to train the bot on and we know that people are coming to that page and asking mostly questions that can be found in those knowledge base articles. So it's very unstructured as far as people are still typing in questions. And you know, the way that you ask a question is probably going to be different from the way that I ask the question. And the LLM is really good at analyzing that unstructured data, really understanding what we're trying to get at and then being able to return information back to us based on those knowledge based articles and its understanding of HubSpot software.

I think that's a perfect articulation of how we got to where we got started. What'd we do, how'd it work? Knowledge give us the, give us the run.

So let's, let's jump into the hypothesis here and what, what we were really trying to solve for and what we were trying to do. So going back to the ISC team, the humans that are handling the chats that are coming through our chat interface. So there are kind of like two types of people that are chatting with us. There's the high intent chatters that we, we were talking about before. And these are people who are showing intent to try or buy software. And then there are people who are lower intent and we're, we define lower intent as people who may not be ready to try or buy software, or people who are just asking very straightforward questions that we actually believe if we were able to use automation to answer those questions, instead of passing people to human to answer those questions, we could actually create a much better experience for that visitor because there would be less time to get to the same result. And so our hypothesis around AI was if we can use AI to truly understand what people are asking for and be able to accurately answer their questions, then we're going to do a couple of things. One, we're going to enable our human ISC team to focus their effort more on people who have more complicated questions that really do need the intention and the back and forth of a human being, or people who have a strong intent to buy, who convert into a customer or a QL qualified lead. So that's our first hypothesis that we're able to do that. And the end result, if we're able to do that successfully would be we should see an improved conversion rate in the people our human ISC team is chatting to and then flagging them as a qualified lead. We'll probably see an increased value per chat. And if we're able to create a really good experience for those people who have more of those straightforward questions, we should see our customer satisfaction score, which we measure across all of our bots, be at par, or maybe even above what our, what our humans are doing, but at least shooting for on par. So that, that was our hypothesis. That was the setup.

We're all curious now what happened.

Yeah, so again, like we talked about last time, you know, we, we trained our model to a certain point, we launched it, we started interacting in the wild, and what we saw was the following. When we first launched the AI bot, we're looking at the customer satisfaction score. And initially we saw that decline, and that was kind of expected for us because we have this new model. We've trained it to a certain point. It's now interacting with real people, and now begins our real work of annotating conversations so that we can start training it to really answer in the way that it should be answering, like a, like a human ISC would. And now what we see over time is that our AI chatbots are on par with our human chats. So we've been able to increase that to on par, and we actually have confidence that we'll be able to maybe even get it above that. So that's the first thing that happened, and that was honestly the most important thing for us, because we want to be able to always provide a really stellar customer experience for our visitors on our website.

By the way, Emmy, anytime you can get automation on par with a human experience, that is a huge win just for anybody out there listening who is testing and thinking through any of this. It's like, wow, that is, that is a huge milestone. There are a lot of companies that would settle for it being like, half as good as a human interaction. Right. So to get it, to get it at parity with human interaction is really remarkable.

Yeah, exactly. We're, we're very, we're very excited about that. The, the second thing that happened was our hypothesis around conversion rates was true. We saw meaningful increases in the conversion rate between people chatters who are being passed to the ISC team and the conversion of those people into a qualified lead. So that significantly increased.

Like, how much? I mean, like a little bit. A lot. Like, give it, give, give the people a little bit more Magnitude on like how big of a deal that was.

That increased by 43%. 43%. So meaningful.

Very meaningful. In terms of lead quality and passing to a sales rep from an automated interaction like that is that, that's literally just free money. Yeah. At the end of the day. Right. And so that's, that is why AI and automation are so powerful and that's why we're, we're doing the show today.

Yeah.

Okay. So we learned a lot. We got it on parity with, with humans and we saw big improvement in results that you just went over. What else?

Let me give you one last stat Kip that we can throw in there, please, please. The other thing that improved pretty significantly was the value per chat. So that was one of the, the core metrics that we were rallying around to see if, if not only we are increasing pass rate, but we are also seeing more value coming from each chat.

Yeah. So it's not just that we're passing more of these to the humans and we're passing bad people to the humans.

Right.

We're actually passing better quality people to the humans and those people are becoming customers at a higher rate.

Exactly. And we saw increases of 50 plus percent. Now that changes, that changes a little bit based on some pages, but yeah, pretty, pretty significant. So meaningful uplifts all around. And the most important part in all of this is we were really able to keep that customer satisfaction at a, at a good level.

And Emmy, keep me honest on this, but from all the data that we've reviewed together in person and Async anytime, one of our thesis around AI has worked and like we've done in a successful experiment, it's never been like a small improvement. It has always been a very, very dramatic improvement. Right. Isn't that, isn't that true?

Absolutely. And that's why this is such an exciting time to be a marketer, because we have this whole new technology now available to us that we're learning, we're playing with and it's giving us these meaningful step function level changes in our core KPIs. And that is, that is really exciting and inspiring. So very exciting time to be a marketer.

Well, this is, this is me like maybe getting on my metaphorical soapbox is there's a lot of marketers out there who are out there optimizing something for like a 12 5% improvement gain. Right. And that over the last few years has been a smart thing to do. Now in the world of AI, you have the ability to build new systems and Programs and automation that can get you fifty, a hundred, three hundred percent improvement gains. So the opportunity cost of going out there and working on something that might be a 5% improvement gain is just far too expensive. Like you cannot afford to do that.

You can't. And going back to, you know, what we were talking about in the last episode with AI, I think there's this, there's this fear maybe with marketers that I need to get it to a perfect place. Yeah, we talked about it before, I start optimizing it. And that will just, that will just waste time for you. So, you know, get it to a certain place, get it out and then you really start getting your learnings and you can see the magic start to happen and you can really see those, those core metrics start to tick up in meaningful ways.

Now let's say somebody's inspired by what you've, you've shared with them today and they want to go and like do this themselves. What would you tell them? Talk to us a little, talk to them a little bit about more specifically what we did, like Kind of paint that picture for us.

Yeah. So Kip, I think it's a few things. You know, at HubSpot, we drink our own champagne. So we're using our chat software. And then we had one person really working on this use case who is going in and annotating the chat transcripts to help train the model so that the model get better and better at answering, understanding what people were really asking for and then answering those questions in an accurate way.

Yeah. So to dive in there for everybody, this is like this person on our team was literally like reading what the user, the user said and what the, and what the AI said and changing the AI response to what it should have been. Like, what would have been a better response, what would have been a better recommendation. And by doing that, that feedback loops, goes back, it feeds GPT. And so the next time somebody asks a similar question, it's going to be a better quality result. And I think we, it took us a couple months of that annotating and tuning to really get it good, to get that customer satisfaction really high. As you keep saying, it's not good on day one. It requires a lot of tuning.

That's right. And now we're, we're thinking through how we can actually. So one of our learnings through all of this is the annotation piece is really one of the most important pieces through all of this. It is also the most time intensive piece.

Yes.

And when we started this we had one person working on it. Now we have a team with a few more AI engineers who are working on our various use cases. But at the time we had one person that was Davidji and he was working on this. And what we have learned since then is this is kind of the thing that slows us down. And so how can we work to unblock this component of training so that we can move faster on our use cases? And what we're working on now is an annotation user interface that essentially allows other marketers outside of this group who we have focused on AI use cases to help us annotate. So eventually what we're going to be doing is our human ISC team. These are the people that are answering the chats. We're going to open up this interface for them so they can actually help annotate for us in addition to the people on the on the AI team. And our hope is with this expansion, we'll be able to move faster, train the models faster, even get the models more accurate. But I think that's going to really unblock us and allow us to go faster.

That's a huge unlock. And so there's a couple things I want to share. You know, when if you're watching the show and you hear a lot about like turbo fine tuning as like a product announcement for a lot of these models, it's basically making it faster and easier to do what we're talking about, the annotating and teaching of the model. And that turns out it's exceptionally, exceptionally important. I think it's one of the things we've learned over the last six months, Emmy, is that fine tuning is a huge, like greater than half of the amount of work necessary to create a great experience. Yes, right.

Yes, it is very important. It's like one of the most important things and it is also one of the most time intensive things.

Yeah, yeah. Very manual, very labor intensive. So anything out there that makes that fine tuning faster and easier is welcome as folks who've been working on this. Right?

Yep, that's right. Kip, last one of the last things on chat, maybe. Do you want to see it in action? Because I would love to see in action.

That's what I was going to ask. I was like, you know, the people, especially our YouTube subs, want to see stuff in action, so I would love to show it to them.

And I think maybe what we could show is. So that that was kind of like our first. The knowledge base use case was our first use case in chat since Then we have rolled it out onto other pages that had different intent on them. But I think one of the most exciting pages that we have rolled this out on is our pricing page. These are in app pricing page. But the, the goal of this bot is not just to be able to field kind of product related questions, but it's able, the purpose is really to help prospective customers discover what products might be good for them and also help them understand the packaging, the pricing and in certain use cases actually enable them to buy touchlessly. And I think that's, that's a use case that we could show the audience today.

Perfect.

I love that. Okay, great. Here's our chat experience and you can see it's saying hello, want to chat about HubSpot? And it even says here I'm an AI sales assistant here to help you find your way.

We've chosen to be very transparent with all things AI we have. So if you were thinking that we're about to drop a question in the comments, we are very transparent.

We're very transparent. The other thing this honestly helps us do and when we first launched AI powered chat, this was very important to us is we wanted to understand the customer satisfaction score. So we wanted to be really transparent and we also wanted to see just how people feel when they're interacting with a bot that they know is AI powered. So, so it really helped us in two ways. So let's just say, you know, hi, I'm a small business. Can you help me understand what package may be right for me? And so the bot returns. Hi there, welcome to HubSpot. I'd be happy to help you find the right package to point you in the right direction. Could you tell me a little bit more about your business and what you're looking to achieve with HubSpot?

Sure.

Right now I keep all my customer data, let's just say in a Google sheet. This is a common use case that we see and I want a better place for.

Yeah, if you're a small business, this is a very common problem, right?

Yes, yes. Okay. I can definitely help with that. HubSpot's free CRM you can store up contacts and companies. You can also connect your Gmail. So it, it, it talks a little bit about free CRM, the value, the.

Kind of that core value prop of the free CRM to this user. Right.

The core value prop of the free CRM. It's speaking to my use case around. I have my customer data in a Google sheet right now and I want a better place for it. That sounds Great. Let's just say I might want to send some emails, too. I'd like to also get a lot.

Of people liking to send emails. So this is. This is. This is right aligned with what we see, you know. Yep.

I'd also like to send some emails and let's see what it says. All right, so now it's saying, yeah, you can send marketing emails from the free CRM. You can send up to 2000 emails per calendar month to get started. You can follow these instructions. And there is a link that it provides.

Right.

To send marketing email. So then it's asking.

So it's going to take somebody right from this into the product to go and do this action, Right?

Exactly, exactly. And so I can choose to go right into the. Into the product and explore that more, or I can continue to chat with this so it can have a better understanding of any other things that I might be looking to do. So it can continue to help me with discovery around HubSpot products and also pricing.

That's awesome. And so you could basically just keep asking whatever questions you want and it will tell you what product fit is. Is right for you.

Yes.

And sales questions are really hard. They're really unstructured. People have a whole host of use cases and problems. Right. So this is something that I think, we think is going to take us a long time to really, really get right. But we're going down the path, and for some of the core, like, questions we get, it's starting to be pretty good.

Yes, exactly.

That is exciting. We've made a huge bet on chat. We are making a ton of progress. As we said just a few minutes ago, when it comes to AI, any thesis that has proved out to be true, we're seeing big gains. Emmy shared that data. And if you are thinking about AI, one of the most obvious places to start is going to be chat on your website, in your product, wherever it might be. Emmy, before we go, we got one. We got something special for everybody. We have. We have an experiment we're about to ship that we want to give a little peek behind the curtain on. And this is about AI and our website.

Yes, yes.

Tell me all about it.

Okay, so we believe, and I think, Kip, you agree with this. I've heard you here.

I mean, I've been to all these meetings, so if I don't agree, I've just been like a really bad leader of managers is what I would say.

Our hypothesis really here is if we are able to leverage AI to better understand what a person is trying to do the job, the job that they're trying to do, the information that they're trying to find. If we can use AI to better understand that and then accurately surface the right content for them to help them get to that thing faster, we will see meaningful improvements in conversion rates. So that's, that's what we're solving for, and that is our hypothesis and the way that we are going about testing this. I can share my screen, but we're building what we call just a website copilot. It is essentially a smarter version of our navigation that is trained on various data sets that is looking to, based on an unknown visitor. Visitor's interaction with our site. It's. It's trying to do just that, predict what it is they're looking to do and surface that conscious them as fast as possible.

Okay, now this might be a little bit like, of a leap of faith for some people to understand. Can we maybe like, see it in action before we even talk more about it? Because I think of all the AI use cases we've talked to, this is probably the least straightforward. It's a little bit. It's a little bit different.

Yes.

All right, so for folks on RSS, Emmy is pulling up a version of the HubSpot website. And on the bottom, it has a AI kind of navigation bar in addition to the kind of static navigation bar.

Exactly. So let's just say that I am an unknown visitor. I've never been to HubSpot.com, hubSpot, we don't know anything about me. I come to HubSpot.com and here on the bottom is this, is this AI navigation? And there's two things here that it says on the, on the left, and it looks like kind of a little pillbox here, but on the. Is this content relevant to you? Thumbs up, thumbs down, and then on the right, there's a little carrot. And next to it it says AI generated recommendations and the number three. So if I click AI generated recommendations, it opens a little frame and there are three pages that it is recommending to me that it believes as an unknown visitor, not knowing anything about me, that that could be relevant. And those pages are our products and features, our customer platform. And then why choose HubSpot? And on the right here, there is a button that's asking me to personalize my recommendations.

I like personalization.

Yes. So let's just. I'm going to click on one of these just to show what this looks like. But let's just say, okay, I. Customer platform I'm interested to, to hear more about the customer platform. So I click on customer platform and now I'm taken to the HubSpot customer platform page and I can, you know, choose to get a demo, get started free. I can read through to learn more about HubSpot's customer platform. Well, let's just say, you know, that's not quite what I'm looking for. I want to personalize these recommendations. So I click on personalize recommendations and then I'm given some information. One, I'm asked, how many people work at your company? And this is to give start to give HubSpot an understanding for what my company like, what my company is, my Persona, what I might be looking to do. The next question is what is the goal you aim to achieve? So there's a dropdown here. I'm going to select researching solutions for my business. That sounds good. And then what is your professional role? I'm going to click on marketing professional because that's in fact what I am. And then I'm going to click update recommendations. And now we have three recommendations. And underneath, underneath each recommendations there's a little indicator as to what percent relevancy the AI thinks that it is.

To me, I like that little like context of how good it thinks our recommendation is.

Yes. Yeah. And it's, it's based on basically it's looking at a lot of different data sources that we have fed it in the background and based on what I am selecting. So I selected a company, 11 to 25 employees. I'm researching solutions. I'm a marketing professional. It believes that the most relevant page for me to look at is marketing software for businesses of every size. And it's saying this is 94% relevant to me. And that sounds pretty good because I'm interested in marketing software. And so I can click on that page and I can now see, okay, here's Marketing Hub. And I can read more about Marketing Hub, the kinds of use cases that Marketing Hub can solve, et cetera.

That sounds awesome.

Yeah.

And so basically, and so for people watching me, we're, we're putting this live for percentage of people visiting the website.

Yes.

And what is our measure of success? Like, how are we going to know if this works or not?

Yeah, good, good question. So it's not, it's not live yet. What I'm showing to you is truly a sneak peek.

Soon. Soon to be live. Soon to be live experiment.

Soon to be live. So it's going to be released to a set number of pages on the Website, the measurement of success here that we're looking at, there's a primary and there's some secondary KPIs. The first KPI is really conversion rate. So we're looking across all different kinds of conversion rate, demo conversion rate, content type conversion rates, et cetera. So that's, that's going to indicate to us, are we actually able to get people to the information that they're looking for as fast as possible and as relevant as possible. And if so, we should see that show up in the conversion rate. Some of the secondary KPIs, though, are more around engagement. If we are truly getting people to the information that they're looking for, are they staying on the site longer? Are they consuming more content and consuming more pages? So those are some of the secondaries, more on the engagement side of things.

Okay, so I think that's pretty awesome. One of the things you and I talked about off, offline on the show, but I think it's an important thing to talk about is like, this is the first little baby step into AI on the website. But if, if, if you're listening to this as we kind of finish up the show here, this has potential, huge ramifications, right? Because you get to a point where you're like, oh, I can see this AI recommendation engine on pages getting really good. And then you're like, well, you know what? Like, why do people need to go to a different page? Why can't we do that, like on a modular level on the website? So that if you're scrolling this page on the website, the model, the, the module on that website changes dynamically based on what you need? And then you're like, well, why do we need pagination at all? Why can't we just dynamically create full pages that are personalized to what people want? And this could potentially be a massive disruption in how people kind of build and use websites, which I think is going to be controversial to a lot of people. All the SEOs out there are going to like, be like, shame on you. They're going to hate everything that you said. I mean, look, they're going to come for me in the comments and they're welcome to. But it's not a crazy inference to believe that that is a path that we are on. No.

And in fact, that that is, that is our vision. If, if you were to talk to Netta Stoll, who's our head of web strategy and leading the charge on a lot of this.

Shout out Netta.

Yeah, yep. Shout out. Nada. We've spent a lot of time talking about where we could go with the website and we've talked about some crazy ideas. You know, should we, when, when AI was first kind of coming onto the scene, we talked about, you know, netta, should we just make the homepage like an open field text box? You know, and then I was like, no, no, no, we can't, no, no.

That'S, that's a terrible idea.

No, we can't do that. And, and the reason is, and she's right. You know, there is a learned behavior that is very ingrained in everyone who has been using the Internet for a long time. And that is you go to a website, there's a navigation on the top and that is how you search for what you're looking for. Dropdowns, you click, you find things, right? So like people aren't quite ready for this, for this huge transformation. But this is kind of the first step in understanding, okay, how, how could we get to a point where we really do understand what somebody is looking for after just the first few clicks on our website? So this is kind of the, the first step in that now a future, a way, future vision. To your point, Kip is, is exactly that you come to a website. And now because we've been through this exercise, we're, we feel very confident that in a few clicks we're able to truly understand what it is you're looking for. We don't necessarily need to have this interface. We could have something different. And when you click on something, because we really understand what you're looking for, the page itself just shifts around to serve to you a one to one experience based on what it is you're looking to get. So we can reduce clicks, keep that experience all in one page and make it truly personal to you.

It's pretty mind blowing to think about that.

It's mind blowing. It's mind blowing.

It's amazing. And it's what we're saying because hey, we're like, we're taking the first baby step down that path. And as we get a little further down that path, you should come back on and share what we've learned. Were we right, were we wrong, whatever. But I think what we do, like, I think we, we do have a strong hypothesis on is that, you know, this free text. We've talked in the chat section about how it's, how chat is so good at handling kind of that free form text chat that's really good when somebody knows exactly what they're looking for and when somebody's ready to chat with a salesperson, they got a pretty good idea of what they're looking for. When somebody's coming to your website for the very first time, they often have no idea what they're looking for.

Yes.

Or a very vague idea of what they're looking for. And that's why we are starting with this kind of recommendation approach. It's like, oh, how, how can we make it just easier and better to like help us surface stuff that is actually going to be helpful to us and then we'll have a bunch of data that'll actually help inform what we build and where we go next. Right.

Can you imagine that, Kip? I mean, think about how many times you go to a website and you just spend such a long time clicking around trying to find what you're looking for. And you know, sometimes you waste so much time and sometimes you don't. Can you imagine a website that can truly deliver to you with accuracy what it is you're looking for, what it.

Is you're actually trying to do?

All within the same ex. Like all within the same page and experience. I think that that is, that is really exciting and I think that's really the future of web design in an AI powered world.

Yeah. Web design is not going away, but it's going to change dramatically and it's going to be even more data powered. And, you know, so having all of your CRM data connected to your website, having all of this training data from interactions on your website is going to be critical in the future to actually building a really great web experience.

Absolutely right. Absolutely.

We talked all about the chat experience driven by AI. We showed you how we did that. The early results, we really doubled down on the fact that any AI experiment that we've done that actually proved out to be true. We've seen big gains, not little gains. Right. And so there's a high opportunity cost in just doing little small iterative experiments. Now that AI is here, we gave you a sneak peek at what Emmy and her team are doing on our websites and how we think that the website experience might change and evolve. Huge thank you to you, Emmy. Huge thank you to everybody on your team who is leading the charge and doing some revolutionary work here. We will keep sharing as we have some awesome breakthroughs. We'll have you back on. Leave us comments. Shout out to Emmy if you want Emmy back on. Always let us know in the comments and we'll see everybody real soon. On the next episode of Marketing, it's the great this data is wrong. Every freaking time.

Have you heard of HubSpot?

HubSpot is a CRM platform where everything is fully integrated. Whoa. I can see the client's whole history. Calls, support tickets, emails. And here's a task from three days ago. I totally missed HubSpot. Grow better.