Copy of SaaStr AI Day: The New Agentic Support Playbook

Recording will be recorded for quality and training purposes.

All the note-takers.

All right, welcome everyone. We're excited. We're back here at Saastr AI Day, um, for a session I've been waiting all day for. Really excited about having, uh, some of the co-founders of Pylon here, Marty and Advith. Uh, Pylon has been a great, uh, sponsor of Saastr for a couple years. Loved seeing them on site, love their energy, and really love the product, uh, that they're building as as the latest launch that we're going to learn more about, I hope, today. So please feel free to ask questions throughout during the chat. Please feel free to send in questions from the livestream. We'll be monitoring those throughout. After Marty's presentation and out of this demo, we'll be getting to Q&A. So again, really excited. Welcome to the SAS for AI Day session with Pylon. Marty, Advith, turning it over to you guys.

Awesome.

So I'm going to go and present my screen now and let me know if we can see that well. Are we good? Awesome. Okay, so first off, quick introduction from myself and Advith. I'm Marty. I'm the co-founder and CEO of the company. We've been around for a little over 3 years. We are around 1,600 customers. We serve B2B companies primarily, and we have always been very interested interested in pushing the frontier of what you can do in customer support. And what the reason we're talking today is last week we did a massive launch that we want to talk about, and we introduced a new way of working for customer support teams that we call agentic customer support. So that's what we're going to talk about today. And then Advith, a co-founder, I'll just introduce him quickly. He's co-founder and CTO, and he will go through a demo of the product in the latter half. So first off, I want to tell a story. And this is going to start not in customer support, but we're going to talk about the one application software that's done really well in the AI era, which is AI coding. And we had an engineer join Pylon earlier this year. And what was special about him was he actually spent the last 2 years on a sabbatical. And so, you know, he, before joining Pylon, you know, before his sabbatical, he was a software engineer, had no AI coding tools.. And then he takes a 2-year break and then joins Pylon. And what he tells us after kind of seeing how everything works was, oh my God, the job is unrecognizable from what it was before. And so it's, you know, the, the before and after is there's like this emotional shift and change in, in how the job is done. And what did he mean? He, he meant for AI coding that before you would do all the work yourself, right? You would write code, you would test code, you would review code, and you'd do it all by hand. To now the world has changed where you're just a manager of agents.

You're just orchestrating agents to do all that work for you.

Do.

Um, and we call this shift— we define it internally as kind of moving to a new agentic way of working. Um, and that's what we mean when we use that word, um, for agentic coding, agentic support. Um, and you can apply it to any of your categories. And so, um, one of the narrative violations is actually when people do think about AI products today, a lot of them are thinking about full replacement of people, which we do see some examples of. But actually, the The fastest growing companies are actually the ones who are doing kind of human plus AI augmentation. And if you look at, first off, even the foundation models like Anthropic or OpenAI, you know, Claude, I use Claude, I use Codex every single day, but it's nowhere near automating me. And if they tried to build an autonomous CEO to start, they would have failed. And actually the company's doing really, really well. You know, those companies I just mentioned, if you look in the coding space, Cursor, or healthcare, Bridge, or legal, Agora, or ARV, et cetera, those companies are all doing humans plus AI types of products. And in the customer support space, it's interesting, no one's actually tried doing the human plus AI model. Everyone's run towards the full resolution agents. So for example, Finn or Sierra that you're more used to, and that works for a segment of your tickets, but it actually works for only, you know, a small amount of the work. And one way to illustrate this is especially in B2B where you are serving customers that require having a lot more context, there's more relationship, you may automate, let's say, 40% of your tickets, 50%, 60%. That percentage may only reflect a small percentage of the actual work being done because what you can fully resolve with zero humans in the loop ends up actually being a limited set of questions. And the questions you do, even if it's a high percentage, might not take care of that much of the work. I was talking to one company recently, they're around 5,000 people and they have around 1,000 people on their support team. It's crazy. And they set up one of these fully automated resolution agents and it deflects around 50% of their tickets, but their headcount didn't change. And the question is, why not? And it's because of this dynamic where you might see on the top end, okay, yes, it can resolve a high percentage of tickets, but turns out those are the easiest ones to solve that actually took up the least amount of work. Actually, most of the work is still being escalated to humans. And helping humans do that work is where we really, you know, focused because we think the automated resolution will be fully commoditized over time. So the new way of working for support, if we follow this old model, and this is really what we introduced with our new Agentic Support launches, before a support team goes and they escalate tickets, they investigate tickets by going to many different tools and gathering data. They might bring things up to engineering or ask them questions. They draft replies themselves to now a new model which we call agentic support, where you are just orchestrating agents, support people are orchestrating agents that do that work for them. So that's kind of that shift that we've built in. And, you know, it's not just the automated resolution piece, it's actually the humans plus AI piece, which no one has done well up until this point. So the job, you know, the same way software engineer joined Pylon, 2 years later, it was like, oh my God, everything is completely different. If a support person joins your company and you've adopted Pylon, they should feel the same way. They should think that same thing. The job is completely unrecognizable from what I've seen before. And one thing to just also point out, this is more for fun category, like looking at what's really taken off so far. There's only one— this is AI spend per department for different functions and really just this top bar, which is AI coding has taken off. Nothing else is even close to that. So we're still at the start. And so, you know, how does Pylon actually accomplish this? We'll look at a demo in a moment, but everyone here I'm sure uses either Claude or they're using chat internally at their companies. And then when they want to do something to help answer a support ticket, they'll probably ask Claude, hey, can you help me answer this question? And they'll connect all of their data sources. So it might be a help center, it might be, you know, a place where feature requests are logged, it might be your support system, it might be codebase, etc., um, or even like account context to understand a customer. And so all of this, all these data sources exist, um, you can crawl them and you can ask Claude to get help. Now this will get you, um, some basic help and it can be very good, but it has a lot of limitations. And one example of this is, hey, let's say, you know, I get a new ticket. One of the best ways to understand how to answer that ticket is by looking at previous tickets that were similar. And if you just build a Claude skill, it's gonna be incredibly inefficient and not practical to go through every past ticket to understand, you know, how we've responded in the past. And so what Pylon's done, we've taken that data, that raw data layer, and we've created essentially context on top so that when you are answering support questions, when you need to quickly reference the account context, we've pre-computed all the setup, the goals, the use cases, the interactions, the sentiment of that customer so that the agent can quickly reference it. And so, you know, for related tickets, for related knowledge articles, for what is the— who is the contact on this account and what do they care about? All of that stuff is pre-computed on our side and then easily accessible. And, you know, when you start thinking about how do I adopt an agentic way of working right now in the market, there are really two options. One is you kind of do it yourself or use Claude. And then I don't know why these are checkmarked. These should have been X'd, but all of these are basically worse. So from an inference cost perspective, Pylon can be like 3 to 6 times cheaper from a quality perspective because we are able to create that upfront context. The quality is far, far better. The speed is way faster. You have team control so you can actually scale instead of imagine a bunch of people all trying to edit or own one skill running in the background. By the time you show up to a ticket, it should be pre-investigated for us. So all of that is true and Pylon's a clear option for that. And then the last thing I'll say is for people who have followed Pylon for a while now, You probably can guess that support is really our wedge into all of what we call customer work. And really our vision here is how can we be an engine for accomplishing work for you related to customer interactions, and then give you a scalable pricing model to actually, you know, pay for that work getting done. And so this is our evolution. We're starting in support, evolving into— we already have some customer success products, but we'll continue to add more and more agentic workflows so you can move more and more work to Pylon. And the outcomes Advith will mention are incredible. Basically scale with the same team you have, have higher quality responses and work getting done, and do everything much faster. So I'll pass it on to Advith to actually show you what it looks like.

Yeah, I'm going to share my screen and hop into the product and just give you a sense for like how, how we internally, for example, use a bunch of these agentic features to actually supercharge our support. Support. So I'm actually going to share now what someone on our support team might see when they look at a ticket. And so on the left side here, you're seeing a ticket coming in from Ryan asking a question about a certain part of our product. He's like, hey, can someone fix this specific filter in the analytics part of our product? It's causing me a pain. And then on the right side here, you basically see Karen, who's one of the support engineers on our team, and what she sees when she's working on this ticket. And on the, on the right side here, basically what you see is our background agent, which runs in the background automatically every time every single ticket comes in. It's looking doing an investigation, trying to look at all of your past issues, your logs, the codebase, any documentation you have, any account context that might have been collected from calls or random internal Slack threads that has kind of tribal knowledge of things related to this customer or your product. It's kind of collected and scanned through all of that information and dumped an investigation on what exactly is going on here and what you might want to do. And so I'll just talk through this really fast. And so firstly, what it's done is it's interpreted the pictures here, understands that's an issue in the analytics part of our product, even though the customer hasn't kind of said that explicitly. It's found two other issues from different customers in the past that are similar. It's looked at a call we've had with this customer last week that where they were specifically frustrated about these issues and it's checked the implementation in the code to verify that it actually thinks the backend for this is implement— implemented, but it's just not surfaced on the frontend. It's gen— given a set of suggested next steps that this is a bug or UI regression and not kind of like something fundamentally missing. It's just a, a quick patch that we need to do. Then Karen can kind of go into interactive mode and ask a follow-up question. So instead of previously having to go check a bunch of different sources, the AI has already done that, but still put Karen in the driver's seat for the judgment on how does it— how do you proceed on this interaction? And so we log all of our feature requests in Linear. And so given that the AI said that this was kind of a missing thing on the front end, she asked, is there an open kind of feature request for this? The AI says there isn't. Then she kind of asks a follow-up question that causes the AI to dig further into the codebase to confirm a suspicion she has. She asks if this is a feature request, whether this should be created as a feature request. She runs a skill which you might be familiar in tools like Claude, which instead of having to write down a bunch of reproduction instructions, why this is important to the customer, figure out the priority of this, kind of refined a skill that she can use where she can just do make linear issue for this please. And it'll go package the customer context, all the investigation that's done, put it into a linear issue and tie it to this Pylon ticket. She then asks it to draft an updated reply to the customer in her tone of voice. It's done that and you can go— you can see that she's gone ahead and accepted this and actually sent this to the customer. So basically the effect here is instead of having to check a bunch of different tools and run manual processes that might have taken a bunch of time, for example, creating a feature request that you might have to do in one in every 5 tickets, you can just do a really quick kind of skill and take the investigation the background agent has already given you and run with that. You're still in the driver's seat and on the hook for kind of the outcome. Like the response is still your responsibility, but the AI is basically helping you take all of the manual steps you would have had to take normally and making it much more streamlined. And so that's one kind of actual example. And I'll click on one more just, just to give people a sense. Here's another example from, from a customer and Owen on our support team addressing his question. Customer basically asked, hey, I have a certain team in our instance of Pylon. Can you let me know when it was deleted? And this would normally basically require Owen to go log into AWS, find CloudWatch, which is our logging tool, write a SQL query, go look up the organization ID of this customer, and then write a SQL query against the logs to try and figure out, you know, what's going on. Then look through the logs and figure out whether this team actually got deleted and what exactly happened. Instead, on the right here, you basically see our background agent doing all of that automatically upfront, allowing Owen to answer this question in a minute or two with just a quick confirmation of looking at these results. Basically what the agent has done here is look through our logs, use the IDs it already knows about this customer and the new one they've provided, to go and find the exact logs. It's already solved issues like this in the past, so it knows exactly where in the logs to look and what the SQL query should look like. And it basically tells us exactly what, what's happened and what the timestamps are for when things got deleted. It drafts a response using that kind of audit trail found in the logs, and Owen just basically sends it as is. And so this basically, it's still Owen in the driver's seat, but it's streamlined the interaction dramatically. You, instead of kind of giving an empty response to the customer saying you're going to take a look, You basically get to give an actual meaningful response very quickly because the agent has kind of done all the dirty work and lifting for you. And in whatever cases you need to, you can ask follow-up questions and go deeper to get more information or take more actions. And so this kind of is a different mode of working where you're basically teaching the agent. The agent kind of takes a first stab, gives you an investigation, and in you going back and forth with it, it's almost like you are writing down every single step you took to solve this issue. And what that means is the next time an issue comes in that's similar, the agent is already educated on where Where do you normally look in the logs? How do you normally solve this type of issue? Like, is there a certain Slack channel on Slack that is about X project that the customer mentioned? Because you've already solved issues in the past using the agent, it kind of improves the quality of its investigation and what it does on each future attempt automatically. So just to really drive home this point of how this introduces a totally new way of working, the idea now is you can work on many different things in parallel. You sort of show up to your support queue and every issue already has an investigation done ahead of time. And then you go in the driver's seat for each one and ask a follow-up question. Follow-up question. For example, I could say, here's an example issue, and I want to say, look in the code base for how we handle email forwarding. And so this is a follow-up question I want to ask to get some more details on how I might want to solve this ticket. I can now click on the next ticket here, and I actually see— if you see this hexagon spinning, I actually see the agent working on, on a previous issue and getting me a result. And while it's doing that, I can go ahead and move on to the next issue, digest what the agent has done on this issue, and then ask a follow-up question and sort of keep moving. And so the idea is it's, it's a fundamentally different way working where you're juggling a lot of different work streams and agents at once, and you're teaching agents through interacting with them how to solve specific issues, and they just get better over time automatically. And so it's kind of an exciting outcome, which I'll touch on in a second, people who are using this, but that's kind of the core philosophy around how people are working now. I'll show one more thing kind of around inside of our product. We've launched this concept called background agents, which basically let you do work automatically in the background. The most common piece of work people like to do is automatically investigating every ticket that comes in and kind of doing the workflow I just showed. But you can set up kind of any number of workflows that are useful. A common one for our team is every time a feature request gets closed, it's difficult for the support team to know like what exactly was the thing that was shipped? Is it already deployed? Does it directly answer the customer's question and kind of solve the, the, their initial pain point? And so we basically built a skill that lets us do exactly that, that runs automatically Every time, every time a feature request closes, we try to summarize what exactly got shipped, whether it's actually deployed and whether it actually fixes the customer's request and just post that in the AI thread on every ticket. And this is an illustration of kind of also using AI to understand what is going on in the system where I sort of just asked it, hey, I know that this background agent exists and this is a process that runs every time a linear issue closes. What exactly does it do? And it basically tells me, hey, this skill that you have set up understands that the customer's original ask inspects that feature request, verifies whether it's deployed, and decides whether or not, you know, you should respond to the customer and what exactly you should say to them. And so you're kind of just putting a lot of the work you used to do manually on autopilot, but you're, you're in the driver's seat for like kind of building some of this process and also actually making the decision on when you want to kind of push the AI along in a certain direction or not. The final thing I'll show really fast is kind of being able to harness a lot of this power just directly in Slack where a lot of people work. And so So here's an example where Nikolai on our team is kind of forwarding a customer issue or a question that he has into the Slack channel. And Linda from our solutions team tagging Pylon asking, can you take a look? Pylon essentially has the same knowledge that it has inside of the app, but in Slack where it can go check your code base, your logs, your past tickets, any customer interactions you've ever had and sort of try and answer your product or customer questions. And so here it's taken a stab at that actual question and given an answer., and then I've hopped in and say, hey, should we update any knowledge base articles so we can prevent this confusion in the future? And if so, can you go ahead and make that change? It finds the relevant article in our docs, makes it, drafts an actual update, and then tags me for review in that update so that I can go confirm that kind of everything looks good and publish the change. And so you can harness a lot of that power directly from Slack, getting answers to questions about your product, getting answers to questions about your customers, like what happened on a call last week? What are, why are they using our product? Or actually just take quick actions related to your customers, like updating your knowledge base articles, or like kind of taking action on an account. So this is kind of, yeah, the power of like working in this agentic way. I'll share some of the outcomes as well really fast if you don't mind resharing the slides, Marty. So customers, yeah, have been basically, we've been in beta with this and a bunch of customers have been using this and the main set of outcomes people have sort of seen is One, kind of dramatic efficiency on their team where they no longer have to escalate things to engineering. They can kind of protect engineering time and just answer a lot of questions themselves. So one customer saw in a pretty short amount of time, about a month, 70% fewer escalations just because they are now armed with a lot of context and they don't have to go ask various people or check various tools in order to like get an answer to a question. The second is just greatly improved customer experience, much faster times to resolution and time to first response. And so another customer saw 64.5% faster time to first response. A big one is scaling without headcount, where people are now able to serve a growing customer base without adding a ton of people on their team. They're able to deliver the same or much better often customer experience without increasing the size of their team. The Slack agent I was showing kind of provides a lot of organizational leverage where now anyone in the team can, can sort of get their questions about customers or product answered. It might be a salesperson who asks questions about the product and often has to go to support or other team members to get that that expertise. It might be product or engineering who wants to know like trends and what bugs are coming up or what feature requests are coming up. Everyone can kind of self-serve that information now. And as the agent is just sort of curating it and serving it up to them. And then a cool thing is this raises the floor basically as a, as a quick last point. It, you might have had someone on your team who was super technical and only they could solve X type of question. Now they can sort of turn what they were doing into a skill and anyone on the team can kind of be guided through how to work through that specific type of, you know, complex technical troubleshooting. This ends up saving a lot of teams a lot of time and improves onboarding a lot. Yeah, those are some kind of a quick collection of the outcomes of what customers have been seeing with a lot of this. And so to summarize kind of the shift from that people are seeing, the floor is raised. It's just a totally new mindset when you're working that all functions are seeing. You're no longer trying to each time solve a ticket from scratch, do the work yourself.— you're really the judgment call. Instead, agents are doing a lot of the heavy lifting for you, and you're directing them on where to look and what to do. Cost is an emerging concern. As we've been interviewing a lot of customers, folks have ended up building a lot of things in-house and have been at the bleeding edge, are now spending a lot of money on, on tools like Claude to try to get the same workflow done. Um, and then people are actually changing how they're thinking about their teams. They're hiring more junior and more technical people. They're often hiring a more dedicated builder type persona who kind of can build this system and, and, and kind of feed the agents, so to speak, to sort of make their whole team run more effectively. So a lot of changes we're seeing in terms of like how people are thinking about how they work and what their team composition is. Cool. Yeah, I think we want to open it up for questions now if anyone is curious about things we've been seeing.

Yeah, related question to what you were just mentioning, which was for the non-technical folks on the team, how much can they impact the agent? Because like a lot of what you've shown is the front end, but like how much is, you know, maybe the non-technical folks on the team impacting the back end?

A lot is the short answer. There's two ways to think about it. One is like just by doing your job and going back and forth with the agent, the idea is the agent is learning from how you solve issues. So if you ask the agent to go check whether there's a feature request for this, it learns that in X type of scenario, you often want to check whether there's a feature request and this is where you check for feature request. So it learns from just you doing your job, if that makes sense. And the second piece would be our job at building the platform is sort of make it easy for even people who are not technical to kind of iterate on the system in the cases they want to. And so yeah, we, we, we have a lot of tooling in the product to make that easy. And it, it is, it is like a kind of a different way of working, but we try to make it as easy as possible for you to kind of, you know, iterate on the agent.

Yeah. Um, another question from the chat is, does Pylon support, um, like SMS in its inputs and outputs of support, specifically WhatsApp, Telegram?

Yes, we do SMS, WhatsApp, and Telegram, so all of those are covered.

Sweet. Cool. Another question that we got was, when it comes to building out this Agentic support layer, how much context should I give my agents?

The short answer is as much as you're comfortable. We try to add a lot of guardrails so that the agent is just, you're still in the loop on everything, if that makes sense in this use case. And so like you still get to verify everything that happens. People end up giving it access to, and a lot of people gave Pylon access to this before too. And so this is not like a newly agent thing of oftentimes your calls, any of the places you communicate with customers, codebase is a new one that in a read-only way with kind of scoped access, you can get a lot of kind of information that's useful in kind of just driving a better outcome where you don't have to escalate as many issues and just the service you can provide to your customers is way better.

Yep. Um, next question in the chat is, can you share any use cases, um, for how this impacts the sales process?

Yeah, I can talk about that.

Um, so our sales team lives in Pylon every day. We didn't actually build it for that use case, but, um, the high-level idea is that we are first off mirroring all the Salesforce or HubSpot data into Pylon. So if you have accounts, opportunities, contacts, you know, all that information, we first mirror things back in. And if you write them into Pylon, they can go into Salesforce and vice versa. And then what's cool is you now essentially have an inbox for all the customer conversation or prospect conversations that you're having at any given point. And those are all tracked across Slack, across your personal email, across, you know, group emails if an email happens to come in somewhere. So it'll track everything and then give you effectively account summaries and overviews of what are, you know, we have what we call like a sales overview view where it's literally telling us What's the timeline? What are the blockers? Who is every contact on the deal? And so an AE essentially has, I think, the equivalent of a Gong summary, but on the account level across all these different data sources. And so it's really, really helpful for understanding like a set of accounts that you might be tracking and then also what's going on with them. And so especially for me as a CEO, when I want to know what's going on with this enterprise deal, I will just ask Pylon and I'll just go in and see, okay, like, where are we? What's going on?

Yeah, super helpful. So it, so it gives them a summary that's basically kind of like a, yeah, after-call summary, right? Of like, okay, here's the support, either ticket or thing that's been resolved. Does it give them like suggested actions of what to do next as well?

So we're building that in. So I showed in the beginning, there's this vision of, hey, starting with support and expanding into all what we call customer work. And the account management and customer success use cases are probably like sooner for us where, hey, identifying upsell opportunity or opportunity to expand with an account will be flagged and then there will be suggested tasks that can be done. And our thought process here is first, you know, create great signals, then create great suggested next steps. And then finally, you know, the ability for you to approve those steps that could be automatically actioned for us.

Awesome. Sounds like some good stuff coming up. One more question is, do you have to be on Salesforce or HubSpot for this tool?

No, you can be on, uh, yeah, we're CRM agnostic is the high level. And then some people effectively call us a post-sale CRM for themselves. Um, I don't recommend not having a, you know, sales CRM. I think you still should have it because from like a prospect perspective, we don't have all the integrations that you would probably want in workflows. Um, but everything post-sale, yes, is the answer.

Awesome. Okay. Thanks guys for doing this. I know we're getting at time here and I want to make sure if folks want to reach out to either of you, I know you're busy as the founders, but How can— what's the best way for folks to get a hold of you?

Um, LinkedIn is great, uh, or if you just want to email founders@usepylon.com, uh, you can kind of hit all of us. And so that's fun. Please don't abuse that.

I like that it goes to all of you. All right, super fun. Thanks for doing this, guys. Always good to have you on. Um, and I know you guys had some exciting stuff just come out last week, so I know— and I know as founders you're very busy. So I appreciate you guys being here at AI Day. You're actually our final session of the day, so thank you for, for taking us home here.

Yeah, of course. Thank you, everyone.

Yeah.

Thanks everyone. See you next time. Bye.