How to Boost Website Conversion with Salesforce Agentforce

Good morning, everyone. Hello, and thank you for joining us. My name is Dana Greenberg, and today we're going to be talking about how Salesforce is reimagining our web engagement using Agentforce. Today, I will be speaking with you. I'm a Senior Director of Product Management at Salesforce. My name is Dana Greenberg. We've got Anant Varma, who is from our Decision Science team, and Emily Cates, a director on our Content Ops team. So salesforce.com, it is a massive website. It's made up of thousands of pages of really rich content. We've got product content, company information, customer stories, SEO articles, you name it. So we were thinking we might be able to use an agent to look across our entire universe of content and bring the most relevant content to our visitors. We could use Agentforce to learn more about our visitors on our site, to have a conversation with them, to help qualify them and understand really what they need in order to drive personalized and curated experiences for everyone. And this could go beyond chat. This could go to our website experience in general, being able to provide truly curated experiences on our site. And so we defined our website agent for salesforce.com. And these are some of the jobs that we prioritize making sure that it did a really good job of. Our website agent is responsible for answering questions about our products and our company, and being able to use that product and company knowledge to drive personalized responses. Our agent can generate qualified leads around the clock. It surfaces offer, either through surfacing offers like a free trial or a demo that someone can be led to and then convert there. Or if someone wants to talk to sales, enabling them to connect directly with sales within that chat. And our agent also needed to be able to operate at global scale. So we do business globally and our customers do their own business in, in different languages and AgentForce needed to work with our customers in their language of choice. Let's take a look at how it works. So on the left, you can see the knowledge sources. All of that content is ingested into Data360, mapped to DMOs, and then indexed for the agent to be able to use. The agent itself is actually pretty straightforward. So it's made up of two topics that do most of the jobs that the agent needs to do, content search and sales transfer, but sales transfer, has an action that can be used to create a lead in our sales org. And it can also perform the job of handing off that lead to an SDR who can actively join the chat. You can see our website with our custom UI that we have for Agentforce. And you can ask the agent any question. Like I said, you could go do this on your own. So we're asking, tell me something, tell me some of the top features of Agentforce. Agent's gonna come back with an answer and you can see that we've designed this more immersive experience. So that we can start to feature a richer conversation. So when I ask the agent, how much is it? What you're gonna see is that the agent can surface a really beautiful experience with snippets from our website, with actual product cards that can help to drive someone down the sales funnel even further, allowing them to eventually, hopefully engage in a free trial. And it's multimodal, so you can see that the agent travels with you throughout the site. But what's more interesting is I'm quickly switching over to Agent Builder on the backend so you can see what's actually driving this. You can see our content search topic with totally straightforward human-readable instructions. And in Agent Builder, you can preview what that conversation looks like and see actually how the agent is reasoning and coming up with a response. So I've asked that same question here. You can see the previewed response on the right. But now what's really interesting is on the left, you can see that the agent reasoned through the intent of that question, determined that we should use the content search topic, and selected the website search action for this question. It's just searching through all of our web page content to provide the best response. What will be different is when I ask, how much is it? Now, with pricing questions, we needed our agent to be more deterministic. So what you'll see here is the agent is generating a response. And this time it's launching the fetch product content action to help provide really deterministic proper pricing information based on our pricing catalog. It's also doing the work that's needed to generate that beautiful card that you saw.

So, hi everyone, I'm Ananth. I'm a director on our decision science team here at Salesforce. I'm gonna talk about how we measure performance of the agent on our website. So the way that we've thought about this is grouping things into 3 key pillars. The first is really all about agent engagement. And this really captures how users are interacting with that agent. So a lot of the things that I had kind of mentioned on that previous slide. So things like, and these are both kind of on a volume basis and then also on rates through sort of that conversion funnel. But agent opens, so how many people are actually opening the agent to begin that engagement with Agentforce on salesforce.com. Conversation starts, so actually typing in a question or using a pre-canned prompt to begin that engagement. And then even things like the number of turns. So between the agent and the user, how many turns are in each actual conversation? What's the average count there? And how is that an indicator of good or bad engagement? The duration of each conversation. Also, as I mentioned before, the pages and paths by which people are using to actually begin their experience. The second bucket is really all about response quality. And so this focuses on the substance of the interaction. And our metrics that we are using to assess how well the agent is responding to users' needs. So we actually developed a custom scoring model. I'll talk a little bit more about that in a couple slides. But basically, on a 1 to 5 basis, it scores every interaction, and every utterance, and every conversation with the agent to give a sense of where the agent is doing well and perhaps where there are areas of opportunity. This scoring model is based on the helpfulness of the agent, completeness of response, actionability, and then it applies to all of the different classifications that we have set on the backend for the agent too. We've also begun to look at some of our key agent observability metrics. So session tracing metrics, things like latency, throughput, error rates, utilization rates, all to get more of a complete sense of the quality of the actual agent response. And then finally, that third bucket really focused more on how we're translating those user interactions into outcomes further down the funnel. So how many of our interactions are actually generating leads for our sales team? And leads we're kind of measuring on two bases. There's direct leads where someone may be escalated to an SDR, which generates a lead in our system. And there's also more of the indirect or influenced leads. And what I mean by that is we saw some of the rich cards that are displayed in an agent interaction, which might drive you to a demo elsewhere on the site, or to sign up for an event or to sign up for a free trial. And so all of these are other conversions on site that we want to be understanding how the agent is actually generating those types of activities too. And so those are more of the influenced lead. This is a view on the response quality side. And so just to provide a little bit more context here, we basically push all of our conversation text from the agent back into an LLM.. And we use a pretty complex prompt that we've evolved over time to then classify each of those conversations into specific topics. And then we also score those topics on a 1 to 5 basis, again, based on helpfulness, completeness, actionability, to see how well our assessment of the agent actually responding to those needs of our customers. So this visual shows how the agent is performing across all of these different topic categories that we've kind of pre-identified using that prompt. Prompts. And we see a breakdown of the average score in that kind of bar across the right, green being best and then yellow being, you know, perhaps areas of opportunity. From all this, we're really seeing some pretty great results already. Just in terms of reach, we have over 350,000 conversations that have been started within the agent experience. It's really a clear sign that people are actually engaging with the agent when they see it on the website and they're clicking it to open. They're actually engaging a step further. But beyond just volume, we're also definitely seeing some really great indicators of quality. The scoring methodology that I just mentioned is showing that over 70% of agent-forced responses are of a high quality, so in that 4 to 5 range. And it really validates that a large majority of our visitors are getting the answers that they need from our agent. And again, we're looking at the low scores very closely to try and identify how we can bring those up and bring this 70% up to 80 or 90. Most importantly, on the right side, Agentforce is also driving real business impact. We've seen over 50,000 leads influenced by the agent, and it proves that Agentforce isn't just answering questions, but it's actively helping us grow the business. So again, we're not just, you know, handling volume, but we're driving tangible business outcomes, generating visitor interest via the agent, and then turning that into real sales opportunities. And on the other side, we're freeing up our human teams to really focus on more complex or high-value engagements. Finally, I did want to call out our new AgentForce observability tool, which is a unified tool for all of our deployed agents with near real-time status and actionable insights. So this tool really allows you to analyze AI agent interactions, drilling into specific topics, segment conversations, and reviewing each moment in conversations to identify potential areas of improvement. It's really a one-stop shop for all of these observability metrics.. So some of the things that I mentioned, like escalation rates, session tracing, accuracy, latency, throughput, all of these are housed in Agentforce Observability. And there's also the ability to build in customizable KPIs with diagnostics or alerts for your specific use case. So we're now actively integrating Agentforce Observability into our— or for our agent on salesforce.com. And it is a part of the broader Agentforce Studio suite. Where you can build, test, and manage your own set of agents.

I'm Emily Cates. I lead our content operations team here at Salesforce. So we're going to go through 4 key learnings. I'm going to unpack each of these a little bit more. But first, starting with clarifying the jobs to be done. Agents have the ability to do so, so many different things that really zeroing in on what is the job to be done, what is the thing that you want to execute on is really key here because you can give the agent access to your whole website. But does it really need access to your whole website? In our case, we really wanted the agent to talk about product and industry information and then hand off to an SDR. So does it need access to our blog? Maybe not. Does it need access to our newsroom? Maybe not. Does it need access to our product pages? Absolutely. Really zeroing in on that scope helped us make sure we weren't adding duplicative information. We had a very smaller data set. And therefore had better results as an output. So that defined scope was really key. And then we were able to optimize the tuning of the skills, the actions, and the topics. Next, we created a cross-functional operating team. And so this is what it looks like for us. It would probably look a little different for you depending on how your teams are structured. But first, these are the key stakeholders. So think about digital. These are the people that have to make sure the agent is on the website. That the agent is working, that people are actually clicking it, that they're advertising it on our website so people actually get to it. Then the analytics. So how's the agent performing? Anath talked a lot about that. So what insights are we learning, are we getting from them, and how can we change our strategy from there? Global enablement. Do people know about the agent? Marketing, in our case, making sure that it was on our website and that people knew about it, they could use it, they could test it, they could trial it. Content operations, what content is your agent ingesting? Engineering for us was making sure we had the right product development or the right product management where Dana's team is looking at, are we building the right things? We're testing with being customer zero. What feedback are we getting? What's working? What's not? So the next thing here, my personal favorite, is managing your content and your data quality. So this is something that I have started saying. It's my new catchphrase. Our digital properties aren't just destinations anymore. They're data sources. So we need to start thinking about them from just being like, okay, someone landed on this blog, or okay, someone landed on this product page. What if they don't land on your product page? What if they just go to an agent? They never see your branding, they never see your creative, they just get your information in an LLM or an agent. And so we need to start thinking about our websites as both destinations and data sources for these kinds of things. When you think about it, content could come in so many different forms. You've got product releases, press releases, guides, blogs, customer stories, and it lives in so many different places. So it's really important to think about, okay, what do I want my agent to do? What data does it actually need? And how am I giving it access to these things or not giving it access to these things? So when you think about content management, here's some questions to ask yourself. When it comes to inventory, do you have a good handle on your inventory? Do you have duplicative materials within your inventory? Would those duplicate materials confuse an agent? Do you need both of them? Who is responsible for keeping it updated? That's a big one. So how often should it be updated? Or should it— when should it be deprecated? Why should it be deprecated? Who owns it? And is this kind of work built into your capacity planning? So is your content team ready to keep all of these things up to date or your web team? Are they on tap to do that? And then how will you measure it? So how will you measure content used by agents? Does that mean that content is more valuable if it's being used by agents? How do you assess the quality of responses? And then how can you write in a way that makes it easier for agents and LLMs to understand? Then lastly, thinking about response quality. Responses are also a form of content in and of themselves. Thinking about how you score these and what actually makes a good response. For us, we thought it was relevance, context, time to respond, and then other indicators. Did someone click on a link? Number of turns, downstream conversions. That might look a little different depending on the goals of your agent.. But let's take a look at this response here. Someone asked, can I build a semantic model in Tableau Cloud that will be used by Tableau Agent? That first little bit says, yes, you can build a semantic model in Tableau Cloud. So there's the relevance there connecting it directly to your question. Then they give you some more context and say semantics is a powerful AI-infused layer in Data360. And then they go on to define semantic learning, conversational analytics. And then lastly, you see for more details, visit this page. So it's giving you that relevance, that context, and then the ability to click into a new page would be an indicator that they want to learn more in this case.