How SaaS Companies are Successfully Productizing AI with Paragon at SaaStr AI Day

I'll just come back at the end to help moderate Q&A. By now, much like everyone else, you got a lot of slides, so I'll let you get at it in a second. So yeah, thanks for doing this. Are you guys ready?

Let's do it.

Good. Okay. Sweet. Let's do it. Okay. I'm going to admit folks. Hello everyone. Welcome back to SaaStr AI Day. We're so excited for you to be here at our next session. For those of you that are going about all day, this is like your halfway point. So you've now made it Halfway through AI Day, we're still bringing you really great speakers and content. So excited to have our next two speakers join us. So Brandon Fu, who's the CEO of Paragon, and Ethan Lee, who's the director of product at Paragon, are going to be talking about how to successfully productize AI. I know they're going to be talking about RAG a little bit, which was— we were just talking about a little bit in our last session. So it's a good jumping off point for those that are watching all day. Excited to have you guys. As always, if you have any questions for our presenters today, drop it into the chat as we go along. They'll try and answer them. If not, we'll share their socials at the end. And with that, Brandon, I'm going to let you take it away and get into it.

All right. Thanks, Amelia. And thank you, everyone, for joining us today. Today we'll be talking about how leading SaaS companies are successfully productizing AI. And as Amelia mentioned, I'm Brandon, co-founder and CEO of Paragon, joined here by Ethan, Director of Product at Paragon. And to share a quick introduction and backgrounds on us, on Paragon. Paragon is the integration infrastructure for B2B SaaS companies. So Paragon ultimately helps software companies integrate their products with the third-party SaaS application ecosystem. Uh, we started Paragon about 5 years ago and have since raised just over $20 million in venture funding and today work with nearly 200 of the leading SaaS and AI companies in the world to help power their integration and product integration strategy. So we've really been a part of how many of the leading SaaS companies have utilized integrations to productize, uh, their AI strategies, and we'll be talking more about that today, starting with some context and background on why LLM wrappers failed as an initial approach to bringing AI to market and how that led to a new strategy of RAG, Retrieval Augmented Generation, and AI agent tool calling. For productizing AI. We'll then talk about some of the challenges of scaling RAG data ingestion and agent tools. And we'll finally share some examples of case studies of leading enterprise AI companies that have successfully implemented these strategies to productize GenAI. So without further ado, I'll hand it off to Ethan to get us started.

Awesome.

Thanks, Brandon. Yeah, just to start and contextualize some of the problems that we'll be looking at today, I want to talk a little bit about why LLM wrappers failed. And really what we mean by an LLM wrapper is sort of, you know, an LLM with a little system prompt that's specific to the type of task that you want the LLM to do, sprinkled on top of an OpenAI or Anthropic API. And essentially what LLM wrappers are, are essentially a reaction to how popular ChatGPT was and you know, the, um, how people realize that, you know, when you're talking to this ChatGPT, uh, bot or LLM, and there's all this power that's coming out of it that seems to be real knowledge, um, you know, people assume that it could do many more things than, than, than, um, than was initially anticipated. So, um, there's a couple problems that people quickly realized that, um, LLMs could not do on their own, and furthermore, LLM wrappers could not do on their own. And the first problem is really that it lacks external context. So if you've worked with ChatGPT for some time, you probably realize that, you know, you have some conversations where it says, oh, I don't have knowledge up until this date because there is a knowledge cutoff date for those foundation models. And really why that happens is that there's a training phase for those LLMs where it's trained on publicly available information from the internet, and it really doesn't have any knowledge upfront about you or, you know, what your business is, what your product is. So what you can imagine is, um, you know, here there are some examples here on the right where, you know, humorously, an Amazon support agent that is an LLM wrapper, you know, you can ask it for generating a React app and it starts spitting out some code in an Amazon support chat. You know, it's just one funny example of an LLM wrapper not exactly having the right context for answering the user's question. Or rather, you know, getting off task about what it's supposed to do. The second problem is that, you know, you can try to inject some data from your application that you have about the user to give it, give the your LLM wrapper some more context. But generally, that isn't a complete view of the world that your user is operating in. So it doesn't have all the business or product context that you might generally expect when you're talking to an LLM app that, you know, is inside of your product. So that's the first problem is that, you know, these wrappers just simply don't have enough context to answer all of the questions that you might want them to answer. Jumping to the second problem, The second problem with this is that wrappers can't really automate work on their own. So what LLMs are really good at is generating text for, for you and answering questions effectively. But without, you know, any ability to actually perform work in the places that you do work, wrappers can't actually set up meetings on your behalf or create records in Salesforce or different systems of records on your, on your behalf. Text generation is not enough, in other words. I think people quickly found this out with LLM wrappers, and that's why people are already envisioning what the next step is for AI in terms of what we are calling agents. Moving forward, we're going to talk a little bit about where the ecosystem is moving more broadly from these simple LLM wrappers that just have this context-specific prompt to really agents that can do work with you and do work for you. In order to talk about that, we're gonna focus on two approaches that people are using to build out agents. The first is RAG, which is, you know, Retrieval Augmented Generation. We'll talk about, you know, why that solves the first problem of feeding in all the context that your LLM app needs. And then the second is agentic tool calling, which solves that second problem of LLM wrappers themselves can't do work on your behalf. So both of these characteristics, having context and also being able to do work, are really important characteristics of agents, and I think will be important more broadly for AI adoption and making AI more useful for your products. So let's take a step back and just think about, you know, some of the places that your context lives and work gets done today. And, you know, these are apps that, you know, apply to you and your team and not just for AI agents, but the same problems still apply. So for example, you know, you have all of your knowledge stored in internal documentation systems like Notion or Confluence Docs. Even in, you know, PDF files and Google Drive and, and, and that sort of thing. All of that knowledge is, is really important for your teams to be able to get the context they need to do their work. You might have important team documents or engineering specifications or requirements, you know, in different places documented in those files. And if your AI agent doesn't have the ability to query those, that documentation, it's not going to help answer those specific internal questions that your team members might have. Second is systems of record. You know, uh, you know, your team today is using, you know, CRMs or project management tools or HR systems, um, in order to track structured information about, um, you know, how work is happening or the status of an order or the status of a project, that sort of thing. And, um, if your agent doesn't have access to those same systems of record that, you know, your, uh, your coworkers do, then it's not going to have all the context it needs to both understand the state of a project, understand the state of an order or a customer. And it's not going to be able to take actions on your behalf like a coworker would be able to do. And then lastly, you know, of course, you know, probably many of us experience this. Nothing is perfectly documented. And the rest of that, all that context that fits in the cracks is in Zoom meetings or Slack conversations, things that are not well structured at all, but are still important for your own team members to have context in, in order to, you know, do their work effectively. So all of that kind of knowledge that's scattered across all of these different apps is important, equally important for your AI agents to understand in order to be as effective as one of your team members and make, make sure that, you know, it can be, uh, a helpful source of information. So to, you know, apply that to, you know, the kinds of, uh, uh, strategies that we can use to make agents more useful, um, there's kind of two approaches to context retrieval or pulling in external information. Um, The first is called, you know, ingestion for RAG, which really what that means is building up a search index for all of these documents and unstructured information that need— the agent needs to be able to query. So for example, you know, we talked about all of those documents that are in Notion or those messages that are in Slack. If someone needs to search a topic by, you know, what are, what are things related to this type of product that, you know, my team needs to look at right now, um, a vector database or a search database is really helpful for an AI agent to ask a question to that database and get all the related documents or related messages about that product and return it to the AI agent to generate an answer. So this is what we call an offline approach because all of that information is being replicated by your app in order for the AI agent to take advantage of it later on offline in a conversation. The second approach is what we call agentic RAG, or it's an online approach, which means that it fetches data in real time in order to pull in information in context for a conversation. So for example, if I'm, you know, asking a Sales Copilot agent about different opportunities that are active and I need to get all contacts associated with that opportunity, I'm going to directly query the Salesforce CRM, or rather the agent is going to directly query the Salesforce CRM to pull in all of that information and bring it back to the agent in context. So that's another important example because in that case there's a structured query that we're looking for. We know exactly what types of context we want out of Salesforce. We don't need to do a natural language search in order to pull that information. So we can talk directly to the API to pull in all that information and get the user's context into the, uh, AI agent conversation. So those are the two approaches. There's ingestion for RAG, there's agentic RAG. Both are used for pulling in context, uh, for the user and solving that first problem of it not understanding all of the, uh, context outside of model or world knowledge. So The next approach that, um, you know, people are using to solve the second problem is really, uh, agentic actions or tool calling. So for example, here's an example conversation where a user is asking to book a call with a contact in their CRM. And essentially what an AI agent needs to be able to do to actually make that meeting happen or to actually book that call is it's going to have to reach out to at least 3 apps in order to make sure that it can book a meeting with the right person at the right time and send all the information to the relevant parties. So, Agentic actions essentially give the agent access to different individual tools that are exposed by Salesforce or Google Calendar or Gmail. So for example, searching for a contact by a specific, um, uh, company name or searching for availabilities on someone's calendar. All of these are individual actions that you can provide to your agent so that it can actually do work in those tools or query information in real time about those tools so that, um, you know, the meeting can actually be booked and that the email can actually be sent, um, not just a draft created for, uh, for your user as a text-based conversation. So in that way, you know, the agent is actually working with you in your context instead of, um, simply, you know, just being this chat, chatbot off in the side that's doing its own thing generating text. So next we'll kind of move to some of the challenges, some of the key challenges that, that we see and, you know, some of our customers share around scaling some of these approaches and what, you know, some of the thorny problems around those approaches. So the first one around ingestion is that it's hard to implement correctly. And here are kind of the key challenges that we see. First is that connecting to third-party data is a big problem and that connecting to each individual API and implementing each individual connector is not only a huge implementation cost for your engineers, but it's a huge maintenance cost for your engineers as well. And we're an integrations company, so we know all about, you know, what it takes it takes to maintain connections, to build out connections. There's a lot of work that goes into making reliable connectors. So that's challenge number 1. Challenge number 2 is that, you know, once you actually have that data, you need to make it effective in order to have the AI agent be able to actually search on that data in your offline database. So, for example, you know, you take a document and you have to actually make it searchable. There's all these different strategies around chunking and vector search that you'll have to fine tune in order to make sure that your agent can more effectively get to the right snippets of information and pull that into its context. So that's challenge number 2 is just ingesting and indexing is a hard project in and of itself. And then lastly, and really what I think is one of the most, um, thorny problems in this is that if you're going to take a copy of everyone's Notion documents, everyone's Google Drive documents, all of their Salesforce CRM records, you better make sure that that copy of the data that you're creating for your app respects all of the permissions and business logic that those original systems of record actually enforce. So of course, you know, you can imagine a disaster scenario where, you know, someone is just chatting with an AI chatbot and is able to gain access to some documents that really, they really shouldn't have access to because, you know, they're accessing a copy of the information that the agent had access to. So, you know, making sure that those permissions are actually correctly enforced by your app and stored correctly by your app so that, you know, the right people have access to the right documents, um, is, is a really, is a really big challenge. So that's one, those are, you know, one among three things to really watch out for ingestion, um, and some of, you know, some of the technical challenges related there. Um, and then moving to tool calling, um, tool calling itself is, is difficult to implement simply because there are hundreds of different actions that your agent will need. Um, it's somewhat difficult to predict or preempt what kinds of questions or tasks that your AI agent will need to do on behalf of users. So you'll have to implement it across a wide range of tools and make sure that the most common behaviors in each of those tools are exposed as capabilities to your agent. I mean, think about all of the SaaS apps that you are going to use just today and all of the individual actions that you're going to take in those apps. Each one of those clicks, each one of those filling out of a form, that's going to be an individual capability that has to be built. Implemented and given to an agent by your engineering team. So, um, you know, it's very difficult to scale out this, you know, large catalog of, um, what we call integration actions. Um, fortunately, you know, at Paragon, we, you know, we actually built out a large catalog of this, uh, of these actions for a separate product for our workflow builder. Um, and we, you know, it, it turned out that these same definitions that we actually built for, uh, for making it easy to construct workflows out of the, these, uh, integrations, um, turns out to be very helpful capabilities to the LLM. And we'll cover a little bit more about that later. Okay, so now I'll turn it back over to Brandon to talk about how enterprise AI companies are leading the way with some case studies. Great.

Thank you, Ethan. And so next, I wanted to share some real-world examples to help contextualize how leading AI companies have implemented these strategies to bring their AI products to market. And so we'll start with what should be a familiar example to many. I'm sure most people here have used Intercom either as a customer or as an end user using their chatbot. And so Finn is the name of Intercom's AI support agent. And so an end user could ask questions in the Intercom chat widget, and Finn would be an AI agent that is able to respond to customers' questions and queries and requests. So early implementations of Finn are a good example of sort of the LLM wrapper problem because the early versions of Finn did not have access to any external context or external data. It really was only able to provide very surface-level or generic responses to questions. So if you asked it a question, for example, like, how do I process a refund, uh, it give, give you an answer that would sort of look like an FAQ answer. You know, here's what you could find from the Intercom documentation. And because it didn't have access to any of specific context about that user, uh, or any of the external data such as, you know, what's orders or purchases that user might have made. Its responses are really limited to what the information is available in the Intercom help desk or FAQ system. So since then, Intercom has integrated Finn with a number of external data sources and actions to really solve this problem. And this is a great example of how they went sort of from the LLM wrapper approach to implementing forms of RAG data ingestion and agentic actions to deliver a far more relevant and useful experience for users. First of all, with ingestion, Finn is now able to connect to external data sources such as support help centers like Zendesk, internal documentation systems like Confluence and Notion, so that it can provide more detailed and relevant responses with context from many different data sources within a company about a company's products and how it works. So it's really able to utilize RAG data ingestion to collect all this context and make available to its agents to pull from to deliver far more relevant and insightful responses. In addition to that, Fin now also has access to tool calling. Essentially, it can make requests in real time to different systems such as a status page, Stripe for billing, Shopify for e-commerce order histories. So it can retrieve relevant data from these different systems in the context of the user that it's chatting with. So you could imagine now if a user might ask Finn about, you know, a return or how they can get a refund, it could actually use its access to these different systems to get data about that user's specific order history, maybe even process the refunds on behalf of that user, all by nature of its ability to integrate and access the context and tools from these different systems. Next, I'll share an example from CopyAI. So CopyAI is an AI go-to-market orchestration platform. Essentially, it helps automate many of the sales and marketing activities that your team might perform today, and it utilizes ingestion and Vagentik automation in a slightly different way, in a different context, but ultimately to deliver a far more valuable experience for its users. For example, it's able to integrate with a customer's Gong meeting recording history to ingest content from call recordings, sales calls, and use its AI, its LLM, to extract important information from those recordings and then perform actions into customers' systems such as their Salesforce CRM to update relevant information in their CRM records. Or send messages to the customer's Slack, to Slack messaging channels. And so CopyAI is a Paragon customer and are today using Paragon to help power many of these integrations to deliver this far more relevant experience to its customers. Another example is TLDB. TLDB is an AI meeting assistant, and not only are they able to transcribe and record customers' call recordings, they're able to also use uses Agentic Actions, uh, tools that its agents can call to take content, uh, from a call recording and similarly perform tasks, automate tasks in its customers' Jira product management system or Trello or Asana, uh, to take relevant information from calls and actually make them actionable and automate the creation of those tasks in these different third-party systems. Last, we have Supper, another Paragon customer who is integrating, or they are an AI-powered business intelligence platform. So what they do is integrate with various data sources such as CRM marketing platforms, billing systems like Stripe, such that users can ask it complex queries about their business and it can utilize the contextual data stored in those various systems to perform complex data analysis and return relevant responses to its users. So you could think of it as essentially an AI data analyst that is able to utilize data ingestion connecting to all of its customers' different systems of record and data sources to be able to provide these complex— or be able to answer these complex queries and provide these far more relevant results based on a customer's specific business data. So hopefully these were helpful examples to highlight how leading SaaS companies and startups are utilizing these strategies today to deliver a successful AI productized experience. And, you know, we really believe that today the main constraints for mainstream adoption of AI is no longer model capability, but really the fact that agents and LLMs by default live in data isolation. Isolation. And we believe that very quickly today the market is converging to a point where agents and LLMs will become default connected to all the different systems, applications that their customers use. And that's exactly the problem that we're solving at Paragon today. So what happens after that point? Once all AI applications are inherently connected, then what are going to be the differentiating factors that separate the winners and losers in the market. So we believe that there's a few things that will be important here. One is response and action accuracy. So AI agents and LLMs can make mistakes. I think we've all experienced that. And so the ability for agents to select the correct tools, perform the correct actions, and access the correct data for a specific context or a specific request is just as, if not more important than its ability to have access to that data and tools in the first place. If it's selecting the wrong tools, for example, it's really not going to be any more useful than if it didn't have access to that data. And of course, we believe that user experience is still going to be a key differentiating factor in AI and AI products. AI should feel like a seamless and magical experience in the context of your customers' existing workflows and the problems that they're trying to solve with your products. And I think what will be important is that products and engineering teams prioritize user experience over sort of the industry trends. And I think many companies today are building chatbots because obviously that's become very popular today. But chatbots are not necessarily the best form factor for solving every problem that your customer might want to use your product for. So I think that, you know, the teams that are most intentional about designing the form factor and the interface of their products, whether it's a chatbot or another form of application, will still be a key differentiating factor that will separate the winners and losers in the AI market. So last, if you are building an AI product, an AI agent, chatbot, we are solving this problem, as I mentioned today. And Paragon is a developer platform for integrations. Today we've helped dozens of leading AI companies productize their AI strategy both by helping them build scalable RAG data ingestion pipelines to ingest contextual data from all of their customers' different data sources, as well as with a new product that we're launching now called Action Kit, which is essentially a single API that will give any AI agent access to thousands of integration tools so that your agents can dynamically retrieve contextual data from any system, perform actions, and automate actions. In different applications. And so if you are building an AI product today, feel free to reach out and we'd love to see if we can help. You can learn more about Paragon at useparagon.com. I'm sure we'll drop some links after our call today, and you can also feel free to email me, brandon@useparagon.com. Would love to chat about what you're building and see if we can help. Thank you, everyone.

Thanks, guys. I think we have time for one quick question. So, Sandeep, let me see if you are still here to ask your question. Sandeep, do you want to come off mute? Okay, maybe I'll ask this question for him. Hold on. No problem. Part of being live. Okay. His question was, do you see the next evolution of SaaS—

where do you—

do you see the next evolution of SaaS tools with an agentic layer that replaces or uses APIs to provide wider context across the org?

Yeah.

So I think agents will certainly utilize APIs, and I think that's, you know, really the means by which they will be able to gain access to third-party systems to ingest contextual data. I think APIs will remain the sort of programmatic interface by which applications and AI agents by extension will use to communicate with different applications. I think the way in which developers implement APIs in the AI and agentic context will look very different. And again, this is one of the problems that we've been focused on solving at Paragon, where with a product like Paragon, for example, you can provide your agents with access to hundreds or thousands of different function tools to call different applications, different integrations, what would amount to implementing dozens, if not hundreds of different APIs. But with Paragon, you would be able to essentially provide a single API call to your agents to be able to dynamically understand, here are all the tools, here are all the APIs that are available to me, and here's how I would use them in the context of a, you know, real-world chat conversation.

Awesome. Well, thanks so much, guys. Unfortunately, we're out of time. So thanks for joining us. I know we'll see you both back at Salesforce Annual this May. Excited to see you there too. Thanks everyone for tuning into AI Day, for watching on the livestream as well. We'll see you for the next session. Brandon, Ethan, thanks for sharing your emails as well so people can get in touch, and thanks for doing this.

Thank you, everyone. Thanks, Aster. Thanks, Jason.