Practical Uses of AI for BDC Managers- Session 3

Okay everyone, welcome into today's session. Very excited to have you here and I know that there are some of you that are still starting to join. We're going to go ahead and get started. Peter has a list of topics that he's going to share and walk through for his third session on how you can use some of these AI tools in your bdc. So Peter, I'm going to throw it right over to you and and we will get started and floor is yours.

Wonderful. Welcome everybody. Third session. This one is going to be a little bit different than the previous sessions and that it will be 100% hands on. I don't have death by slide deck today, so you're welcome, or I'm sorry, whichever you prefer. But today what I thought I would do is cover a very important topic. And incidentally, and not maybe not coincidentally, we are also working on adding some of this functionality into the Volley application so that you don't have to do so much work. But I think after today's session you'll see that what I'm about to show you to equip your agents and equip yourselves with better real time information at your fingertips and at your agent's fingertips. It's not really that hard. Some of it takes some time. And so some of the exercises that I've done today, we won't be doing it real time. I've actually done them in advance so that we don't have to wait for the LLMs to do their processing. But all of it is, you'll see my chat history. I'll walk you through step by step exactly how to do each of these steps. I'll walk you through in Volley. Some some of you have already put an incredible if you're a Volley customer and you're running a BDC or if you're an agent at One of our BDCs, some of you have an incredible wealth of information already in the Volley application and I want to walk you through where we've seen our customers use this in the application and then how to get that into the LLM so that you can take full advantage of it while your agents are on the phone calls, while you're working through any other problem sets, et cetera. And it just today's will be all about enabling knowledge management within your bdc. And there is an incredible amount of data that you have collected. I know some of you are using third party tools to manage your dealership's profile, for lack of a better term, the hours of operation, whether or not you can schedule oil changes on a Saturday, what your loaner policy is for Mercedes vehicles, what your loaner policy is for recommended maintenance, etc. All of you have different ways of cataloging that information. Some of it is within the Volley application, some of it is on third party tools. I've seen some of you, when I do site visits, use Notion. I've seen some of you use your SharePoint, Microsoft SharePoint. I've even seen some of you use Jira or Atlassian Confluence, which is a a knowledge management platform. No matter where you have the data, there are many, many ways that you can use AI to leverage it. I think today's session won't go the full 90 minutes. In fact, I'm confident unless we have a bunch of interaction, which I hope to but you I'll probably let you out early, probably in about 50 minutes. Unless there's just a ton of questions at each step of the way. I'm going to pause for any questions that you might have. So Jake, be ready at the moderator front to to bubble up any questions or comments that come in. And with that said, the goal of today, again just in summary, before I share my screen and we get started, is to enable better, faster, more efficient, more accurate knowledge management and knowledge dissemination between your dealership, your your dealer group and your agents as they're on the phone with customers.

So and as you're pulling it up, we did get a comment in the chat already asking if they are hearing double on you speaking, wanting to know if it's just them. So open it up real quick. I might be a little echoey, I'm in a new space so apologies, I haven't got all my rugs and stuff to absorb that sound but is just okay, sounds fine. Okay, we're good then. So okay, thank you very much. And Peter, before we dive into anything and if you're going to cover this, just say hey, we're going to cover that. Can you take a minute and I see some names on here that I recognize that this is their first time joining one of these sessions. So can you take a minute or so and explain what is an LLM? Because we're going to throw some of these terms around and abbreviations and I want to make sure everyone knows what we're talking about.

Sure, it's probably a good idea just to recap kind of where we've come the first session we spent a large amount of time going through what a GPT is, what our current implementations of AI in sort of general release are out in the market space and what LLMs are and then also prompts and prompt patterns. So just because like Jake said, there are some of you that are joining for the first time, all of you, unless you've been living under a rock, in which case you didn't get this webinar invite, but all of you have seen either in the news, your kids have used it, you've used it ChatGPT or Facebook's Meta, I guess it's Meta's Meta, AI now Claude AI, etc. All of these are example of what, what are called large language models. And a large language model is a way of. It's a technology that allows you human chat like interactions and as we talked about last time, even voice interactions with these AI tools. And a GPT just stands for generated generative pre trained model. The LLM stands for large language model. And both of these are forms of AI, they're both related. An LLM what its sole goal is they get trained on, let's just call it the Internet at large. So just like Google, these LLMs are constantly going out and able to, some of them are able to access the Internet in real time, some of them only have training data up to a certain point in time. But what they, what they attempt to do is understand the full catalog of all of the human knowledge that is available on the Internet at large. Just like a search engine, but then the technology and the algorithms behind them. Large language model refers to how they understand what you're typing and then how they generate a response to you. So in a lot of ways, when you boil it down, it's not really artificial intelligence, it's more like augmented intelligence. An LLM sees the words that you're typing, they understand those words, they compare it to what they understand as their data set, whatever that might be, whether it's on saltwater fishing or making blueberry muffins or how to study for a college exam on US history, whatever. They compare what you're typing against those and then they compare it against the knowledge set and what they're trying to do. Every word you type and every word they respond with is predict the next word that they are that you are typing and predict the next word that they are going to respond with. So they're not sentient by any means. They're just comparing using very advanced searching, sorting and predictive models, they're just comparing what you're asking or what you're saying with the typified most likely response based on its training data. We discussed in one of the in the first session, the Mary Had a little lamb episode, LLMs are non deterministic and that's just a fancy programmer speak for the outcome is not going to be guaranteed. So you put in Mary Had a Little Lamb and as we saw in the first session it generated most of the time it would generate the rest of that poem or that child limerick, but in some cases it would add things and it would also respond with sometimes humor. And if we did it enough, it might respond with some sense of irritation. And so that's all the LLM is. What we're going to be doing today are what would be classically referred to as prompt engineers. Prompt Engineering. Bless you, Jacob. And the prompt, as we talked about in the first episode, a prompt is simply what you type in to one of these LLM interfaces. And there are various patterns that we can use to take advantage of the technology that underlies these LLMs. Patterns that were designed in from the beginning based on the predictive and qualitative analytics that the code does that built the LLMs in the first place. And so these prompts are just ways of us taking advantage of the underlying technology and algorithms that are built into these LLMs. I'm certainly not smart enough to understand exactly how they work, and I'm maybe even less qualified to explain exactly how they work to you, but in layman's terms, it's an attempt to model human language and interaction within a software application is what an LLM is, what the patterns are, and what the prompts are.

That sounds great. Great as background and let's dive into today's agenda.

Great. So I'm going to share my screen and on my screen is going to pop up the Volley Communication Center. All of your agents use this every day. All of you have taken some time to pre populate the Communication center and all of your campaigns with various scripts and templates, as well as policies on your dealer and any other information that you want your agents to know while they're on the phone or texting or emailing back and forth with your customers. This is our demo account. I'm going to be showing you just a few brief things on our demo account, just so that we get our bearings, so to speak. In the center tab we call it internally, we call it the Kitchen sink on the dev team. And it's just our place for putting anything that doesn't fit neatly into a bunch of other categories. When the agent clicks on more here they have all of these different categories or ways of organizing information. All of this is information that either you've put in directly as their manager or team lead or whoever set these up, as well as information that we've pulled from the various parts of the customer's interactions with your dealership, information that you put in, etc. So today we're going to focus on one specific thing in here and we are currently on the organization tab. But if you have promotions, a tire promotion, an oil change promotion, etc. Any external links, any other things that you've put in here, you can access that and we can use that within these LLMs and within this, underneath this umbrella of using AI as a knowledge management tool and a Q and a platform FAQs, however you want to term it. So on the organization tab you have the just the basics of each of your dealerships. If you're running a dealer group or if you're an individual dealer, you have your dealer in here. So this is just the basic, I don't know, we'll call it contact information of your dealership. And so you've got the name, address, phone number, etc. And then underneath here you can have a yarborough of information and you can put in things like your loaner policy, how to log into Xtime or your other schedulers, what your hours of operation are, who your parts department manager is, what they need to do for recalls. There's a whole bunch of stuff that you can put in here. And a lot of our, our customers, our accounts have put in an incredible depth of information here for the agents to look over, so much so that the agents may not be able to find it when they immediately need to. Additionally, on the communication center you have the ability to add one or more what we call survey templates, but they're just scripts. And a lot of you don't use them for scripts in terms of words that you want the agent to say, but you use this as a way to categorize different parts and pieces of information that you want your agents to be able to quickly access. So you can have any number here, right here I've got just a script that we've populated with a bunch of different service operations and why customer may need them. And last time one of the things that we covered was using the LLMs or AI to generate rebuttals and then doing rebuttal trainings. Those are oftentimes found here, but in this case we've got a bunch of why should I do this for the customer? So if they ask, well, why do I need to do front and rear brake service or why do I need to rotate my tires, why do I need to get an oil change every 5,000 miles, etc. Etc. So you've put this content in here, but again it's not immediate to the agents. And so what we're going to talk about today again under this umbrella of knowledge management is how in the heck do we make it so that our agents and get the benefit of using the LLM while they're doing their job, while they're on the phone SMSing with customers. So with that said, what I've done here, just in the interest of time and making sure that you're just not watching me copy and paste and type is I've put together a number of documents for today and I'll show you how we're going to use them once we get to the LLM. So the first thing that I did is I created a list of my dealership information as well as all of my staff directory. So I've got their name, their title, what department they're in, their phone number, their phone extension and their email address. So that's in this first document along with my hours of operation along with some information here. So this information up here, all I did from the Volley Communication center is I copy and pasted this organization information out of here and then using some queries I generated some fake information about our fictitious dealership Sunrise Motors of Fort Myers where it's always sunny. So here's my contact information, here's my dealer sheet. So that's the first document that I've created and I want to show you how you can use AI to very quickly and simply give your agents the ability to search and ask questions of this data. So what I'm going to do here is I'm just going to copy this and I've got Claude up which is an LLM and I'm just going to open it in a new tab and I'm going to again some of the prompt patterns that we discussed earlier, I'm going to give it some information. This is sometimes referred to as like the boundary pattern. I don't know that it has an official pattern name but here's my prompt that I'm going to type in. And Jake, one of the feedbacks that I got from the last session is to, to copy and paste and keep track of all of the stuff and all of the prompts that I use to generate this. So what I will do after this is I will put those all in a document, all of the prompts that I use and their responses. And I will send that to you, Jake, and you can provide that as part of the show notes on YouTube when we publish this as well as to all the attendees. So don't worry about copying and feverishly typing or screenshotting these prompts as I type them. I'm going to cop, I'm going to put these all in a document and we will provide that as part of the follow up after this show. So you can just sit back and relax and watch the sparks fly here. So all I'm going to do, so remember I've copied this information out and I'm going to say I need you to help me create an FAQ for my agents. It. So just reading through it, I've told it what I'm going to do, I'm going to, I told it how this data is going to be used. And then the key thing here, because a lot of these LLMs have access to the current Internet, meaning if they don't find the information in their pre trained data, they can go out and search for it and ingest it on the fly. In this case, we don't want them to do that. We want to limit them to only the boundaries that we have prescribed. Some of you, whether it's in the news or in articles or talking with other people, have heard about this thing called hallucinations and a hallucination. Again, remembering how LLMs work. All they're doing is trying to predict the next word that you're typing in and the next word that they should respond with. A hallucination is simply when it doesn't find the data, it's going to generate perfectly acceptable language in response to you. It just doesn't have to be true. So on a recent podcast that I was listening to from a gentleman that created a language learning product, they said we don't care if the AI hallucinates, we just care that it does it correctly in the language that the user is trying to learn. Which I thought was really interesting. So it's not teaching them history, it's teaching them how to speak Spanish. Similarly, the AI, if you start asking questions that it doesn't know the answer to, it's going to just make things up that are grammatically correct. And so the key part here is this. That is the only information that I want you to reference. So I've asked it to read through this and let me know when it's ready. So again, think of this as your agent is on the phone and they are being asked questions and in this case maybe they need to reach out to or provide contact information about who's at your stores, if they're a new agent, they don't know them by heart, etc. So it's ready for me now to ask questions. So I'm going to try to drive to questions about this particular person who is a fictitious warranty in administrator at this particular store. So if I'm the agent and I'm relaying the customer's question, so let's see what it says. So I am a customer and I want to know whether or not the appointment that I'm about to make with your agent is going to be covered under warranty. So there with a quick question, I've been able to say these are the two people that I might want to ask questions of. I can ask other questions. So here it referenced not only the text data that was in there, but it also found and return relevant results on the dealership's hour of operations. So again, rather than having an agent scroll through all this giant list, they can just relay the questions that the customer is asking or that they are interested in. And within just a couple minutes we put that data in there and we were able to give your agents an FAQ tool that they would be able to ask questions of at the time of the call, et cetera. So this is just a very easy way for you to streamline your agents, streamline your phone calls. Maybe it only saves you a couple seconds on each phone call, but multiplied by your 30 agent times five days a week or six days a week, it adds up to a lot of time and it also adds up to a lot more accurate answers for your customers. So I'm going to just pause there and Jake, any questions from the field?

Nothing in yet, but I have a question that I've heard other people ask before in different scenarios that I just want to make sure that we address. So when I log on to ChatGPT or Grok or any of these tools, and I've seen this, other people ask this as well, how do you make sure that it saves this information rather than starting a new set session when your team pulls it back up?

Sure, there's a couple of different ways to do it. The first way to do it is to just supply your agents with a single document. That way there's a single point of reference that they can put in as their first prompt of the day. You can always start a new chat. The other thing that any of these LLMs allow you to do is you can see your chat history. So these are all the recent things and you can star them. So an agent could have one for every different type of information, the historical chats, et cetera, maybe they start one for every single customer. The other thing that you can do on some of the LLMs is you can create custom models with pre trained prompts. And that's a topic that we're going to be covering on the last, the fourth session of this using your AI in the dealership. So we'll get to that specifically. That is really a, I'll call it a technical deep dive. And we will get to that on the next session so that we can combine everything that we've learned so far about LLMs and AI and we will actually create a custom GPT in ChatGPT so that all of that data is already there with some caveats. But the best suggestion for now is, is create a single document that has all of this information. Again, if some of you are using Notion or SharePoint, that's really easy. And the next exercise that we'll do is showing you what happens when there's just too much data. But that's how you do it. Jake is either a custom GPT which is sort of the Capstone project, or you can just provide it in a single document that you keep updated and then provide out to your agents and you can put it as an external link if it's on a SharePoint or you could provide it as a shared document or another type of document with whatever document solution you're using in house.

So just to ask this back then we will talk about creating the saved model, so to speak, next time. Yes, for today's purposes, and this is actually a question we had in the chat, I'm telling you, I've seen this question. So the idea then is for agents to have an LLM open with the document and then at the start of the day refeed it the document every day. So that's the idea for today, to get it so you're not in today's session and we'll cover how to get it to save and build upon next time. But for today's purposes, we are starting fresh each time. The idea is create the document that you then feed into it and you're back up and running after 12 seconds.

Yep. Yeah. And again, you can always reference your previous chat history. And because the LLM keeps track of your conversation, I'll use that term in quotes because the LLM keeps track of your conversation when you use a previously stored or starred chat from the LLM, it remembers all of the types of questions and it actually uses those in reference to the answers it provides. So you can do it either way. You can upload it fresh every day in a new chat, or you can use a save chat. Depending on the LLM, they store them indefinitely. Actually, depending on your plan and the LLM, they either store them indefinitely for up to 30 days, for up to a week, et cetera.

Sounds great. No other questions at this time, though we did have someone ask that one. I'm telling you, I can read all the minds of all this. So it is a skill. We have had a couple people click the raise hand and immediately stop it. So I imagine we'll have a. They were clicking the button. I imagine we'll have a question or two come in. But with that said, why don't we keep moving for now and then we'll answer them as they come in.

Okay, so I'm going to move on and I'm going to, we're going to go to the next exercise because this is something that I hear all the time from my internal team. I hear it from other people. I hear it from. If I'm talking with people who are using the LLM in their personal lives, whether it's to help their kids with their algebra homework, et cetera. I hear this all the time. And I'll tell you what it is and then we'll get to how to solve it. The LLMs all use tokens as a matter of their currency, and we talked about in the first session what a token is. Think of it, for all intents and purposes as a word. And all of the LLMs have a number of tokens that you can submit to them, and a number then they will respond to you within the confines of, of the various models and plans that are within that LLM. And you will periodically try to either upload something or generate a response that exceeds the token limit. And it'll tell you we ran out of horsepower or you ran out of tokens. Please wait. In the case of Claude, they make you wait a certain amount of time until the token. Their algorithm effectively releases tokens back into the ecosystem. With ChatGPT, it'll just tell you I've hit my limit. Or what often happens, especially with really large data sets, is it will just fail and it'll just stop processing. And it'll tell you it stopped processing because it's just effectively run out of memory or tokens on that particular request. So I'm Going to walk you through this example and this is a super helpful illustration of one of the principles we talked about in the first session, which is when you try something, don't stop trying if it doesn't give you the answer that you want, keep trying different angles until you get the answers that you need. And so we'll walk you through this. So I'm going to share my screen little disclaimer as soon as I share it. Walzer Automotive Group is a large automotive dealer group in the area in which I grew up and Volley has no official affiliation with Walzer. I'm just using this as an example, this is not an endorsement from them, etc. It's a public website, but it serves as a good illustration for how you might run into problems in this exact sort of knowledge exercise. So I'll walk you through what I did here. I started on ChatGPT and I said on this page, this one, which has 27 different rooftops of various makes and models. On this page I want you to give me a list of the dealership names along with their address, phone number and hours of operation in markdown format. And just really quickly, markdown format is just a way of formatting text files. It's it's oldest time and it's just a way of creating, you know, headings, bold bullets, etc. And it's a really convenient way of, of dealing with data when it comes to the LLMs because they understand the format. So remember on the site originally there was 27 dealerships. So it says here's a list of automotive dealers, including their names and what I asked for. So I've got 1, 2, 3, 4, 5. So it came up with five dealers, not what was on that site. And then I tried it again, except this time what I did is something a little sneaky and I'll show you what I did in Chrome. This web page loads in Chrome. And what I did is I simply saved it and I saved it. You'll see there's options here. I can save the whole thing, I can save it as a single file, or I can save it complete, which will create a bunch of different folders with all of the other components. I saved it as just the web page. And so I'll bring that up and I'll show you what that looks like. So this is the Walzer web page as loaded from my local machine. And you'll see because I didn't save it complete, it doesn't have any of the images or any of the, you know, Some of the formatting is missing, but I did that for a reason. So the LLM allows you to upload data. So this time I said, here's this file. Again, I saved it right here on my desktop. Here's this file. Now, I want you to do the same thing, except this time, instead of going to the URL, I uploaded the file and I just told it, on this file, this is what you should expect. And so it created a CSV for me. And here's the CSV that it created. And on that one, it looks pretty complete. To be candid, I didn't check it perfectly. And when I yelled at it about it, I made a mistake. And I didn't count all five. I said there were four. That was all that was on my pain at the time. So it says, here's a structured list of walls, of walls or dealership locations. If you'd like me to download it in markdown or downloadable file format here, click it here. Well, when I downloaded that, it wasn't in markdown, it was in CSV. So ChatGPT, again, on the second time, didn't quite do what I asked. So this time I said I wanted a markdown format. It lists out a short number of dealerships. And I said, there's only four now. And it intuited that by me saying There was only 4, but in the list there was 27. I want you to do something different. And so it processed a little bit, and then it gave me the entire list of dealers. So that file is right here. And this again is in markdown format. And that's not critical. I would suggest that as you provide data to these LLMs, both in and out, you use something that doesn't create, that doesn't contain a ton of formatting. It just makes it harder for the ll. It seems to make it harder for the LLM to really find what you want. And it does perfectly fine with just text. It is able to, again, based on the algorithms that undergird the technology of the LLMs. It does perfectly fine, just like a human would, saying their sales department is open here, here, and here does just fine. Understanding that data. So now that I've got it in markdown format, now I can do some other things. So then I went back to Claude and I just uploaded that markdown file. And I said, again, I want you to use this file and this file only to answer questions. Please let me know when you've read the file and are prepared to answer questions. So when I uploaded that, it took some time. And that's why I did all this beforehand, is because I didn't want you to sit here and wait and for you to have to tolerate my dumb jokes that I would use to fill the silence. So it says, I've read it and now I can start asking questions. So I asked it a question. How late is the service open at the Bug GMC store? It says, well, actually there's two of them. And here are their hours. Can I schedule a service appointment for my 2012 Silverado on Saturday at 7pm based on the information in this file, you would not be able to schedule a service appointment. And here are the hours. So again, by just doing a little bit of trickery with the LLM to get around those token limits or to stay well within the. Well within them, I am able to upload it a ton of data and I'm able then to have my agents get valuable information at the time of communication so that they don't have to guess. They, they don't have to get it wrong. Whenever I go on a, on a visit to one of you all or to any of our other customers. One of the most common things that I hear is that an agent made a mistake. They, they scheduled an appointment when they should have, shouldn't have. And now the BDC manager is getting yelled at by the service advisor or the service manager for a mistake that their agent made because either something wasn't clear or frankly, they just made a mistake. This is one way that you can use to help help your agents better serve your customers. Get to the information that you're looking for because I guarantee you've collected a ton of it. This is an easy way for you to provide the LLM valuable information for your agents at the time of these phone calls. And I'm not limited to just hours of operation. We can talk about policies, we can talk about certain scheduling scenarios like recalls, et cetera, and any document that you can put in to the LLM. The LLM is more than capable of reading it and then allowing your agents to access it at the time of their phone calls. So I'm going to pause there before we get to our third and final walkthrough. Quit sharing. And Jake, any questions this time?

No, not at the moment, but everyone's very engaged and, and ready to keep moving.

Okay, so the last one is kind of a variation on the theme. It's just more of the same. But I wanted to use it as a demonstration to talk about that third and final category that we talked about. Within the Volley communication center and that is your, your various operations. So I'm going to start a new chat here, going to go back to this and all I'm going to do is I'm going to copy and paste it out and I'm going to give the LLM some boundaries here. It so taking this, I pasted this all in so it says I'm ready. There you go. So customer objects to tire rotation or they object to some other service. Now I'm going to ask it a question that I'm pretty sure isn't in here and we'll see what it does. So it immediately broke the rules and it went out and it started searching the web. So here we go.

It.

There. So now again this is a perfect example of when you don't get the information you first want. Sometimes you just have to try multiple approaches and it's also a way of ensuring that the information is correct and the information that you want is by constantly setting boundaries. One of the things, one of the reasons why in the next exercise we will discuss building your own GPT is you can build in a lot of these controls so that it's just less for your agents to think about when they're on the phone and communicating with customers. So again I can answer, I can put this all in a single document, my hours of operation, my contacts, policies, my objections handling my rebuttals, etc. I can put that all in one document, feed it to the LLM as a document so that you get around some of the token limitations and then use that to ask questions again. The most common reason we hear between friction between a BDC manager and the service department is agents set an appointment that they shouldn't have. And so hopefully these tools in part allow you to reduce the likelihood of that happening and make your agents lives easier, make your lives easier, etc. That's all I got. That's the third example. I guess I'll just kill the screen share and see if there's any questions.

No, that's great. I have a question here off the bat, you know, we all, I'm sure have done, you know, varying levels of varying amounts of playing around with these tools and you know, inputting certain, you know, the, the dumb example of non serious work type thing that Peter likes to say is, you know, give me a recipe type prompt. But just even take that one question I have as you apply it to your work life is what tips do you have for, you know, as you give these lists and prompts to Your agents who may not have all of the information that the management team does. And sometimes these models get things wrong. Still, you know, the tech is good. The tech is a great tool. But there's a reason why we at Volley believe it. These are agent enhancers, not agent replacers. And that is not the only reason. But one of the reasons is because they're still not perfect and they'll probably never be perfect. So I guess my big broad question is what advice do you have for people looking to implement these in a professional setting to help avoid some of the, as you called it earlier, hallucinations?

Sure. There's a few things and I'll just kind of enumerate them. The first is start with good source data. That's the number one thing that we see is starting with good source data, data that you have refined in your store as policies that you've collected manually and you've put into digital format. I go to a lot of accounts where the agents, their cubicles are smattered with different printouts and, and, and other things that they reference on calls. I would say the number one thing that you can do is start with really good source information. That's, that's the first thing. The second probably goes without saying, although, you know, we have a saying in our house that you can't teach good judgment. And you know, sometimes you, your agents are going to just be required to deploy a modicum of common sense, whether it's, you know, loaner policies, whether it's scheduling constraints, et cetera. You know, some of these things, you, they need to accept the LLM's answer as probably true, but trust, but verify sort of a thing. I think the third thing, as we talked about in last time's session, is you can use these documents and you can use these exact same techniques to develop training quizzes and interactive type exercises for your agent so that rather than having to chat them every single time and look them up every single time, they're committing it to memory. They're getting used to some of these things. And that allows you to, in a training scenario, have them practice and not have the customer on the line in terms of, you know, making an upset customer or making promises that the dealership then has to bill. So those would be the three things I think the, the first and the last are probably the easiest to implement because again, can't teach good judgment, but using the source documents that you trust and, and you've, you've built over the years and then doing this in practice have the agents Practice doing this in their one on ones with their team leads or with you have the agents practice asking questions and then verifying the answers with you. It's just a really good practice in any scenario. We don't want the agents just reading this stuff off blindly. We want them to have great conversations, make these customers their own friends and, and make it very conversational. And the only way to do that is with practice.

Is it fair to say that? And those are all great tips and certainly practice is king with all of this stuff, with anything. But is it fair, fair to say that for all intent, these aren't perfect. So still prone to some errors. But for all intents and purposes, if you feed it data, hard copies of materials that you have put together, you are better off than better off at avoiding some of these mistakes that they make than if you were to say, hey, search the web for this. Searching the web is where these often kind of pull things out of nowhere, right?

Yes, of course, the dealership two doors down, maybe a Maserati dealership or maybe a Lexus dealership. And their loaner policy is likely vastly different from yours. So that's why it's so important to confine them. I still think that the LLMs have a long way to go. On a recent podcast that I was listening to is the people that are that are building these things estimate that the are going to be about 200 times not not percent, but 200 times better within the next five years. And that's a level of interaction and comprehension that frankly I don't really understand. They're getting better every single day. I've heard that hallucinations in current chat GPT comparing it to the first model that was released, are down by almost 100%. So they hallucinate very, very rarely, but they still hallucinate. You know, if you do a million chats, 1% hallucination rate is still an awful lot of interactions. So trust but verify. It's going to be probably better than than 80% of people's knowledge, but not as good as your best agents. It just isn't.

We have a question come in here. Is the AI able to read the information off of an Excel or Google Sheets doc, for example, we have a doc for each store that lists recall numbers with corresponding instructions for booking in rows and columns and such. Or would you recommend formatting that into a text file?

The answer is yes. The yes to which because this or that yes, yes, it's able to read it in probably whatever format that you have it in. The caveat being that a lot of times it will, in my experience and other people's experience may vary when you feed it a Google Sheet. And the first thing you have to do is you have to make that wide open because it has to be able to access it publicly. So just because you can access it with your Google account doesn't mean that the free world can. And in order to access it, it has to be able to see it on the web. So that's the first caveat. The second that I've run into is that it doesn't read the whole thing. Whereas if you in Google Documents or Google Sheets, if you simply export it as a CSV or comma separated values sheet, then you can upload it to ChatGPT, Claude, etc. And it will read it and import it natively and you can ask questions. It's the same with PDFs. So I have a lot of our accounts that have taken scanned images. You know, I referenced the agent with documents taped up all around their cubicles or they have a notebook with, with printouts or whatever. I've, I've had a lot of accounts and helped several that they scan all of those PDFs in chat, GPT, Claude, etc, they're all able to read what's in a PDF. Now it's not perfect, but it's pretty darn good. And again, it's just a really quick way of you getting that data in there. The best is text. Second best is CSV because it can write code to actually import it and understand them. And then the third best would be PDFs and images or things like that.

And so just to, to back up to the prior question then with the Google Sheet format, even if it did access the file and it was going through, you're still prone to, it's looking up on a website that is not local. So you would recommend download the file even if it's still in a spreadsheet or you know, a CSV or a Excel. You would recommend that over here's the link to the Google Sheet.

Yes, affirmative.

Okay, perfect. Well, that was the question. We've had a couple thank yous posted for the session, but any other questions? We're going to hang on for a minute or so. I think that this was a great session and I hope that all of you are starting to think about some of the ways that you can use these tools to level up your processes. You know, a lot of this stuff isn't volley specific, but Peter really wanted to share some of the things he's been learning, if you can't tell, we at Volley are very, I want to say it's stressful how much information there is and tools and things we need to learn, but it's all very exciting. It is an exciting time. It's a lot. But by implementing some of these things, we can all level up our games. So we hope that you're finding this informational and helpful. So, Peter, any, any closing thoughts? If not, we'll give folks the the time back.

Well, a couple. The first is I really appreciate everybody joining. I really appreciate all of our customers who are on these as well as anybody that's interested in becoming a Volley customer. We are here for you. We want to make Volley great for you. And so some of the functionality that I've gone through on all of these sessions, my dev team is diligently working on bringing some of that functionality into the app so that you, your agents don't have to go out and you don't have to go outside of Volley. Until then, you know, hopefully these webinars are really helpful to you. I've extended this offer before and I've had a few of you take me up on it and I'm dead serious when I say my email address is peteroli.com if you have any specific questions, want examples, want me to jump on a zoom etc with you and we can walk through and I can help you together. I am more than happy to do it. I do it for you as a customer. I do it for you if you're not a customer but are attending these webinars when this goes up on YouTube. Maybe this won't age well in 50 years, but I will just make the genuine offer again. If there are any specific help that any of you want, hit me up. Email me if I'm going to be in your area on a on another customer visit, I'll come see you and we can do it face to face. But please don't hesitate. I really mean it. I'm here to help you. I want you to do better. I want you to make more use of these tools and that's what we're here for. So that's my final closing comment and we appreciate all of you.

Sounds great. Well, thank you all very much for taking the time. Reach out if you have any questions or want help with anything and we hope you all have a fantastic day and we look forward to seeing you on the fourth and final session where it sounds like we're going to be learning how to make a dedicated model for some of these that will save your data and kind of grow alongside you. So should be a great session. We hope you all have a fantastic day.

Thanks, folks.