Practical Uses of AI for BDC Managers - Session 1

Okay, didn't forget. Well, welcome in everyone. You got Peter and I on here today and it looks to me like we're wearing the exact same shirt. This was not. Are you wearing the blue, the blue volley shirt?

I'm wearing long sleeve.

Ah, darn. Darn. We almost matched. So let's go ahead and give it another 30 or 60 seconds and then let's get started here. We have a lot of people still joining, but this should be a good, a good session here today, don't you think?

Yeah, I'm excited. We have a bunch of volley people. I'm going to teach them how to cheat on their jobs as well. So this should be fun. I hope it's fun.

Good. It should be. Should be a great session. Let's get started here with, with today's session. Peter, what I want to do here.

How to cheat on their jobs as well. So this should be fun. I hope it's fun.

Well, good. It should be. Should be a great session. Let's get started here with, with today's session. Peter, what I want to do here is just throw it over to you to to kind of get things kicked off, but I want everyone here who's in to know that as Peter's going through, feel free to put any is just throw it over to you to to kind of get things kicked off, but I want everyone here who's in to know that as Peter's going through, feel free to put any questions that you have in the and we'll get to those either throughout or, or at the end we'll have them how to.

Cheat on their jobs as well. So this should be fun. I hope it's fun.

Well, good. It should be. Should be a great session. Let's get started here with, with today's session. Peter, what I want to do here is just throw it over to you to to kind of get things kicked off, but using AI in your bec with Volley CTO Peter Sperry. Peter, take it away.

Well, thanks Jake. Really excited to be here with all of you today. Just for kind of a backstory on this and some of you saw the video that Jake and I did late last year about the, the genesis of this, this four session series is that when we started to look at how we could use AI before, both internally at Folly and externally in the product, we had to do a lot of learning. AI was relatively new to the chat GPT launch of course made everybody aware of it and what I didn't want to do as one of the people in charge of what our product is and a lot of the policies that we have internally. What I didn't want to do is just jump right on board and make a lot of mistakes that early adopters tend to make in, in any new technology. And so what I did is I waited for a while and waited for the tool set to really get to a place of maturity. And then I said I would never do this again. After I graduated from college, I said I'd never go back to school. Well, I ate those words and I went back to school. I took some extension courses that were offered by a couple of the universities that I had attended in the past and, and really immerse myself in what is an LLM, what is AI, what does it even mean? And then also how to take advantage of it, how to use it for small things and big things. And as some of you know, that are either closer to us internally or externally. We've built out an AI team now as part of the Volley development team that is solely focused on using AI in the product. But we get enough questions from our dealers and frankly I've gotten a lot of questions internally from our folks where they are saying hey, you know, I'm trying to use chat GPT to do X or I'm trying to create some sort of an agenda or maybe an email series, etc. Is that something I could use AI for? And so enough of these questions came up where I thought maybe everybody could benefit from that, that. So this is a four course seminar, webinar, etc. The next one will be in late April. Look for scheduling on that to come to your calendars. Today's is really going to be probably the most, most heady one, although we'll try to make it interactive and fun. But really what I wanted to focus on today was, was just kind of the basics and, and then we will go from there. And Jake, any questions or anything we need to do before I get started or should I just pull up the, pull up my slide deck and we can get going?

Yeah, I think, I think that's great, Peter. I think that we should pull up the slides and get rolling and again, if any questions or anything come up, put them down below. And we appreciate you all taking the time here. Like Peter had said, this really is meant to be a, you know, an educational value presentation for you. You know, this was kind of something that came up. You know, like he said, we did that video at the end of last year teasing this, you know, this, this was just something that Peter wanted to do because he, you know, he's learned a lot and he wanted to share it with you all. So it's not, not any more complicated than that. And Peter, I see you got your slides up. Let's roll.

Yeah. So titled it Using AI in your bdc. Really what this course will focus on is a particular segment of AI which is the LLM or large language model. So we'll go through. Here's a course outline. I didn't do any fancy animations, so for those of you who came expecting that this would be a whiz bang slide presentation, my team internally knows this. This is about as far as and cool as I get. So first thing we'll go through is what are LLMs? Second thing quickly what are the most popular LLMs? Third, how do they work and why is it important? Maybe to understand a little bit about how they worked. And then the the bulk of today's session will be spent on prompts and prompt patterns. So first thing is what is a prompt and then what are prompt patterns? And the last thing we'll do is just more of an open ended thing. I hope that this gets pretty interactive with the team and Jake will rapid fire me out how they work and then the the bulk of today's session will be spent on any questions that come up. But what are some ways that we might be able to leverage LLMs in the BD on prompts and prompt patterns. So first thing is what is a prompt and then what are prompt patterns? And the last thing we'll do is just more of an open ended thing. I hope that this gets pretty interactive with the team and Jake will rapid fire me see or in the dealership as a whole. If there are repetitive tasks or data munching that you're doing on a repetitive basis, maybe we could explore some of those. In future courses we will go into depth on things like script generation, rebuttal generation, agent training, etc. But just as a, as a way to caveat any questions that come up. But what are some ways that we might be able to leverage LLMs in the BDC or in the dealership as a whole if there are repetitive tasks or data munching that you're doing on a repetitive basis, maybe we could explore some of those in future courses we will go into depth on things like script generation. Rip off today's session. I wanted to open it up at the very end to, you know, brainstorming. You got a brain buster that we can maybe solve interactively. So hopefully you come armed with that as bottle generation, agent training, etc. But just as a, as a way to cap, we're going through the first part. I hope that you all write some things down or maybe have some examples that we could do together. So just a quick disclaimer off today's session. I wanted to open it up at the very end to, you know, brainstorming. You got a brain buster that we can maybe solve interactively, so hopefully you come armed with that. As we're going through the first part, I hope that you all write some things down or maybe have some. In no way, shape or form would I propose to be an expert on this. I have done spent a lot of time in it, both on the prompt and LLM side as well as on the development and integration side. So I know a few tricks. But you know, I'm not opposed to learning new things and I certainly realize that every day I, I'm humbled by what, I don't know, examples that we could do together. So just a quick disclaimer. In no way, shape or form would I propose to be an expert on this. I have done spent a lot of time in it, both on the prompt and LLM side as well as on the development and integration side. So I know a few tricks but you know, I'm not opposed to learning new so without any further delay, what are LLMs? So I won't read this for word for word, but I'll, I'll get probably pretty close. LLMs are large language models and LLMs are a type of AI and they are designed to process and generate human like things. And I certainly realize that every day I, I, I'm humbled by what I don't know. So without any further delay, what are LLMs? So I won't read this word for word, but I'll get probably pretty close. LLMs are large language models and LLMs are a type of AI and they are designed to process and generate human like language. So most of you on this call have probably done some experimenting and use ChatGPT. If you have kids that are in school, they've probably tried to use it to write a paper or do their homework. But really the power of LLMs is that they are conversational, so they are trained. So all of the LLMs that are out there are trained on multiple data sources and they are trained on an language. So most of you on this call have probably done some experimenting and used ChatGPT. If you have kids that are in school, they've probably tried to use it to write a paper or do their homework. But really the power of LLMs is that they are conversations credible number of data sources, so vis a vis the entire Internet or the publicly available Internet, any other source, documents, et cetera. And what this does is it allows them to learn patterns of human language and it allows them to inference context within that language as well. So they are characterized by just having huge scale. So right now in the United States, it's estimated that by the year 2032 that there will not be enough energy production in this country to fuel the needs of training large language models and training AI. That's how expensive and how expansive the training of these models are. So some of these models, Facebook Meta, for example, has multiple models and they are trained on billions of parameters. What it does is it allows them to capture things like nuances of language and sarcasm, humor, grammar, syntax, semantics, and then how we put sentences together. Some key features of the LLMs language understanding. They can understand or comprehend and interpret human language, including context, tone and intent. And we use some of this for our products internally at Valley. And so some of you who have seen some of our demos on a product we call Pulse would be familiar, familiar with some of this text generation. So because they are trained to understand human language, they can also generate output that mirrors coherent and contextually relevant human text. And often it's indistinguishable from human conversation language translations. So LLMs are not just trained in English, they are trained in multiple languages and understand the cultural nuances of putting languages together. It's not just literal translation, it's oftentimes contextually and linguistically correct formulations of alternative languages outside of English. One of the things that we're working on at Volley is the ability for agents to automatically translate text messages or incoming emails into a language that the customer or they themselves are more familiar with. And then they're conversational. This is a huge one. So often when you are approaching interacting with an LLM, you'll do it in a very conversational tone. In fact, a recent study by OpenAI found that most of their interactions, over 50% of their interactions use words like please and thank you when they are talking to the ChatGPT interface. So they use human relevant. And the way that you would talk to a human is the way that people speak with them or text with them. I guess the most popular LLM by far, CHAT GPT. Chat GPT took about a month to get to the level of scale that Facebook got to in five years. So it had incredibly fast Adoption it was the first really to publish and popularize this language and chat model for interacting with the LLMs and GPT stands for generative Pre trained model. And really all it is is it's a chat based interface for one of these large language models. And so they made that method of interaction popular. Claude, which is a LLM that was developed by Anthropic, it is a very predominant tool in creatives as well as software developers. So Claude is used almost universally on our own internal team here at Volley. Claude is being very aggressive about capturing the development community for doing things like writing code, writing tests, writing documentation, et cetera. But also for writers who are, and content publishers like, you know, search engine optimization and website optimization are using Claude to generate very compelling text literature, et cetera. Llama is a product that is open source by Facebook or Meta and it's incredibly widely used in research and in mobile because it's open source. So Meta has released Llama under an open license so that you can use it for both commercial and non commercial purposes and you can host it internally. So this could save you some money. But really what it saves you is if you want to develop custom models or custom training for LLMs. Llama is a really good way for you to do that. And they also have really good image generation, all just perfectly integrated into their UI and some of their apps. Grok is a somewhat recent entry, although you know, who knows what's been worked on for how long. But Grok has become really popular over the last six to 12 months, mainly because you know, Twitter is as pushing it, Elon is pushing it. It's also kind of funny, one of the things of the, one of the nuances and you can host it internally. So this could save you some money. But really what it saves you is if you want to develop custom models or custom training for LLMs. Llama is a really good way for you to do that. And they also have really good image generation, all just perfectly integrated into their ui. And some of their apps of Grok is that they wanted to make it understand things like sarcasm, it references previous conversations and, and makes jokes. And so we'll go through some of that later on today. And Gemini, Gemini is a product that, that was released by Google. Gemini is integrated into a bunch of the Google platforms and suites. Some of you who use Gmail have seen it, some of you who use Android devices obviously have used it, et cetera. Grok is a somewhat recent entry, although you know, a few of our developers internally use Gemini on their Phones to do things like call screening, etc. So Gemini is Google's entry into the space and who knows what's been worked on for how long. But Grok has become really popular over the last six to 12 months mainly because, you know, Twitter is pushing it right now. You know, you would say well which one do I use? And the answer is it probably depends. Elon is pushing it. It's also kind of funny. One of the things of the, one of the nuance, all of these companies and all of these products are in an arms race and you know, the, the, the, the winner is the one that probably has the best and most trained models, but there will be a lot of competitors in this space for the next who knows how many years of Grok is that. They wanted to make it understand things like sarcasm, it references previous conversations and, and makes jokes. And so we'll go through some of that later on today. And Gemini, Gemini is a product that, that was released really as it kind of shakes out, you know, you'll have to decide and some of some Google Gemini is integrated into a bunch of the Google platforms and suites. Some of you, who of them do better at tasks than others. So it's probably not a question of which one, it's which ones. One of the ones that I did not include on here is a company called or is a LLM called Mistral. It's also one that you can host internally for research purposes and that's what it's used for. A lot we use Gmail, have seen it, some of you who use Android devices obviously have used it, et cetera. Here we get into a little bit more of the nitty gritty. But you know, a few of our developers internally use Gemini on their phones to do things like call screening, et cetera. So Gemini is Google's entry into the space and right now you would say, well which one do I use? And the answer is it probably depends. People often think that an LLM is some sort of magic behind the scenes and when you break it down into how it was trained and how they were built, they're not all that complicated under the hood. And that's minimizing it of course quite a bit. But they're not some black magic voodoo that's going on. This is all software code that's been written by humans and with the existence of AI. But what a large language model does is it is a token based system for predicting the next. All of these companies and all of these products are in an arms race and you Know the winner is the one that probably has the best and most trained models, but there will be a lot of competitors in this space for the next who knows how many years really as it kind of shakes out, you know, you'll have to decide and some of some word that you are going to type is effectively what it comes down to. So when you start typing something in, it is already pre processing that and trying to predict what your sentence is. And then similarly when it's creating outputs, it's doing it by predicting the next best word for it to insert based on the context of the question that you put in, based on its training data and based on some of the techniques that you might use to elicit a response to it. So it uses a transformer architecture. So when you have them do better at tasks than others. So it's probably not a question of which one, it's which ones. One of the ones that I did not include on here is a company called or is a LM typewords. It puts those words into tokens and it basically called Mistral. It's also one that you can host internally for research is it's predictive modeling on those tokens. They do pre training purposes and that's what it's used for a lot with all of these models. So again they're processing millions, trillions, quadrillions of tokens in order to understand what, how humans talk, understand information that's out there on the free Internet, also understanding research documentation that maybe isn't part of the public domain. The better the LLM is trained, the better the output and then parameters. Here we get into a little bit more of the nitty gritty. But you know, people often think that an LLM is some sort of magic behind the scenes. The parameters just refer to how many items was this LLM trained on. So with Facebook's meta, depending on your application, you can choose which model you want and when you break it down into how it was trained and how they were built, they're not all that want. Some of them have as many as 90 billion parameters, some of them have as few complicated under the hood and that's minimizing it of course quite a bit. But there's 4 billion parameters and it just really depends on the outcome. So just last thing and then we'll get a lot more interactive, I promise. When you give an LLM a prompt first, the thing it does is it tokenizes your input so it takes a common sentence. One of the things we'll do is go through Mary had A little lamb. And it's breaking those into tokens it processes. They're not some black magic voodoo that's going on. This is all software code that's been written by humans and with the existence of AI. But what a large language model does is it is a token based system for predicting the next word that you are going to type is effectively what it comes down to. So those tokens through its neural network again trying to assume that the next thing that you're going to say and it calculates the probability of those distributions for what token should come next. So again it's building sentences as you're typing and then building its response as as you're typing and as it's responding so it selects a token. When you start typing something in, it is already pre processing that and trying to predict what your sentence is. And then similarly when it's creating outputs, it's doing it based on these probabilities. It adds that token to the sequence and then it repeats this process. Some of you have seen there's a little spinny wheel as you're typing into chat GPT. That's what it's doing is it's calculating what its response should be. Key mechanisms by predicting the next best word for it to insert based on the context of the question that you put in based on its training data and based on some of the techniques that you might use to elicit just this will be part of the slide deck. But the last thing is probably the most germane things that are response to it. So it uses a transformer architecture. So when you hear so LLMs don't understand what you're saying. It's not another human, it's not another sentient sort of being. It's all probabilities and type words. It puts those words into tokens and it base math. So they don't understand what you're saying. But they are a sophisticated pattern matching software that really have become incredibly good at misit's predictive modeling on those tokens. They do pre training with all of these models. So again they're processing millions, trillions mimicking how a human thinks and converses. So just to kind of illustrate this we will go through just a really silly exercise but just to point out how quadrillions of tokens in order to understand what how humans talk and understand information that's out there on the free Internet. Also understanding some of these LLMs are are interpreting what you're saying. One thing to note about LLMs is they're not deterministic so the same input won't always generate the same output. And because everything that you're doing is trained search documentation that maybe isn't part of the public domain, the better the elevating it on what the next output should be. So if I type in Mary had a little LLM is trained, the better the output and then parameters Lam it's already, as you can see down here, coming up with what it thinks on the parameters. Just refer to how many items was this LLM trained on? So with Facebook's meta, depending on your application you can choose which model you are going to type and then it's going to think about its response and it just finishes the want. Some of them have as many as 90 billion parameters, some of them have as few. This popular children's limerick for me Mary Had a little lamb there's 4 billion parameters and it just really depends on the outcome. So so again we're coming up with different things. Just last thing and then we'll get a lot more interactive, I promise. When you give an LLM a prompt it first the thing different responses. And if I go Mary had little it does is it tokenizes your input. So it takes a common sentence. One of the things we'll do is go through Mary had a little Lamb and it's breaking those into tokens. It processes them. So here chat GPT is giving us a little bit of humor. It can see that I've already repeated this twice, so it uses my conversational history in it. You might say it's getting fed up with me, but it's continually learning and it's composed tokens through its neural network again trying to assume that the next continually evaluating our previous conversation for what it's about to do next. So it's not understanding what thing that you're going to say and it calculates the probability of those distributions for what toke I'm doing. But it is generating text and we'll do it one last time to see what she wish you what it says and should come next. So again it's building sentences as you're typing and then building its response. So now it's giving me the whole thing, all the verses. So response as as you're typing and as it's responding so it selects a token. This exercise, it shows you two things. One, that it takes the conversation and the context into account as you're working through the interface based on these probabilities, it adds that token to the sequence and then it repeats this process. Some of you have seen There's a little spinny wheel as you're typing into Chat GPT and the second is that it's not deterministic. It's not going to give me the same output every single time based on the inputs. And this is critical and that's what it's doing is it's calculating what its response should be as you're talking about doing things maybe in your BDC or maybe in your dealership where you're going to be giving the LLM some context. And I'll key mechanisms give you an example from the product that we're. That we have Volley working on right now. Just this will be part of the slide deck. But the last thing is probably the most germane thing here. So LLMs don't understand what you're saying. It's not is a product called Pulse and Pulse is a call evaluation platform. So it is evaluating every phone call as it's coming in and then produces an evaluation as an output. And what we found when we were developing another human, it's not another sentient sort of being, it's all probabilities and mapping Pulse is that the same phone call, the same transcription wouldn't always give us the same results. And so that was things that we had to train the model and that we had to train our product. So they don't understand what you're saying but they are a sophisticated pump generation to account for when we are doing these evaluations that the same phone call won't necessarily and so we had to give the LLM boundaries pattern matching software that really have become incredibly good at mimicking how a human thinks and converses or how we want that evaluated. Therefore we got better results and better outcomes. So just to kind of illustrate this we will go through so back to Slide Deck the next section and the the remainder of our time together will be spent on prompts. So what Just a really silly exercise, but just to point out how you saw me just type into the the ChatGPT user interface was a prompt. Some of these LLMs are are interpreting what you're saying. One thing to note so a prompt is an input text that's given to the AI system that guides and instructs the AI on what to generate and respond with. It's effectively the starting point about LLMs is they're not deterministic so the same input won't always. Some of you have heard of things called prompt patterns and it's not the case where you can just always ask the a always generate the same output and because everything that you're doing is training it on what the next output should be. So if I type in Mary had a little AI or the LLM to do something for you or give you an answer much in the same way that Google Search right you when, when you go into Google Search, typically you are formulating in your head what are the phrases I need to type in here in order to get what I want. And some of you have learned some tricks like putting things in quotes or maybe referencing a specific website. If you only want Google Search to consider results on a specific website, those are sort of tricks or hacks on how to hack the Google search engine. And the LLMs are no different. Embedded within them are certain logical and software patterns that effectively allow you to get and refam. It's already, as you can see down here, coming up with what it thinks I'm going to type and then it's going to think about its response and it just finishes the this popular children's limerick for me Mary had a little lamb find better results out of the out of the LLM. And so a prompt pattern is just a structured template or it's a structured approach for crafting effective prompts for large language models. And what these patterns do is they help you get better results. So a piece of feedback I've gotten almost universally if I'm on the plane or, or if I'm at a family get together and people ask what we're working on or people ask what I do and I talk to them about AI and they'll say something like oh, I tried that chat GPT1. So again we're coming up with different things and it didn't give me what I want. I, I don't think it works. And oftentimes most of the time the problem is not with the, the different responses. And if I go Mary had little so here Chat GPT is giving us a little bit of humor. It can the fact that it didn't work or it didn't provide them good results is that they didn't crack. See that? I've already repeated this twice so it uses my conversational history in it. You might say it's getting fed up with me but it's continually learning and it's continually evaluating our previous conversation for what it's about to do next. So it's not understanding what I'm doing but it is generating text and we'll do it one last time to see what she wish you what it says after prompt or take the time to understand Maybe there's a way to hack this. And so a prompt pattern are just that. And so we'll go through a few of them and you'll have to forgive me here we'll kind of flip back and forth between the LLM chat GPT and we'll use Claude a couple times just so that you can see the differences. But we're going to flip back and forth here as we go through the prompt pattern so that you can see examples of these. So direct prompting. Direct prompting is you are asking the LLM to produce some sort of an output and that's it. It's. They're usually pretty direct. This is mostly analogous to putting something in the Google search bar. You're searching for something, you're asking for something. So we'll go here. So now it's giving me the whole thing, all the verses. So I have to stop this exercise. It shows you two things. One, that it takes the so a direct prompt is I want you to create a recipe or a vegan conversation in the context into account as you're working through the interface. And the second is that it's not deterministic. It's not going to give me diet in the Greek probably has to be capital and Greek cuisine. So this is an example of a direct prompt. So we've asked it to do something. We haven't really put any boundaries on it other than it should be vegan and it should be the same output every single time based on the inputs. And this is critical should be of the Greek cuisine. And this is what it outputted some as you're talking about doing things maybe in your BDC or maybe in your dealership where you're going to. You have done this. Some of you have done something like I have these five ingredients in my fridge giving the LLM some context and I'll give you an example from the product that we're that we have Volley working on right now. What can I make? I caution you on those, but sometimes they turn out okay. So this is an example of a direct prompt role prompting. This is a really powerful one and it's probably one of the simpler ones that you can use to start helping get better results out of what you're asking the LLMs to do. So role prompting you are telling the LLM to pretend is a product called Pulse. And Pulse is a call evaluation at someone in giving you your answer. So I'm going to say act as a call grading expert in my call center and evaluate the things my agent is saying in platform so it is evaluating every phone call as it's coming in and in this phone call and then you would then produce as an evaluation as an output and what we found when we were developing pulse is that the same phone call and the same transcription wouldn't always give us the same fight it what your agent said maybe it's a transcript of the phone call maybe it's something else agent says to customer in results and so that was things that we had to train the model and that we had to train our problem we don't have any cars in stock generation to account for when we are doing these evaluations that the same and so you can see here Chat GPT is telling us it's it's stepping into this role and now it's going to pretend in its response that it is phone call won't necessarily and so we had to give the LLM boundaries for how we want that evaluated Therefore we got better results and better outcomes that expert call grader and it's going to provide you with so back to slide deck an answer based on that Persona. Another way that you can use this is if you have ChatGPT is telling us it's it's stepping into this role of kids, okay and they're in school act as a third grade teacher and now it's going to pretend in its response that it is that expert call grader and it's going to provide you with an answer based on that Persona. Another way that you can use this is if you you're gonna My typing skills are on display here and the next section explain the US Civil war and the the remainder of our time together will be spent on prompts. So what you saw me just type into the the ChatGPT user interface was a prompt of kids, okay and they're in school act as a third grade teacher. So again here you're going to My typing skills are on display here and explain the US Civil war. So again here I'm telling Chat GPT I want you to pretend that I'm that you're so a prompt is an input text that's given to the AI system that guy something and so in its response it's going to give you a response that's tailored. I'm telling ChatGPT I want you to pretend that I'm that you're something and so in its response it's going to give you a response that's tailored I do these exercises with my kids sometimes and instructs the AI on what to generate and respond with it's effectively the starting point. My son's 11 middle daughter is 13 and my oldest is 16. And if they're struggling with something at school I will have chat GPT maybe create a study guide for multiplying or dividing fractions based on me. Some of you have heard of things called prompt patterns and it's not the case where you can just always ask the my children's grade or age and it's going to come in it's going to explain or the LLM to do something for you or give you an answer much in the same way that mean it in ways that in this case a third grader could understand. So the Persona pattern is extremely powerful for you. If you guys are if, if all you on this call are giving maybe reports or or Google search right you when, when you go into Google Search typically you are a breakdown of something to executives you could say act as a dealer principal or formulating in your head what are the phrases I need to type in here in order to get what I want. And some of you have learned some tricks like putting things in quotes at my dealership and critique this proposal for me and it can provide you feedback in the same way that an automotive dealer principal would be able to give it to you. So the Persona pattern is extremely powerful. It's probably maybe referencing a specific website. If you only want Google search to consider results on a specific probably one of the most powerful and simplest techniques to master because all you need to do is website those are sort of tricks or hacks on how to hack the Google search engine. Think of the Persona that you want mimicked and have it craft its response accordingly. Chain of thought and the LLMs are no different. Embedded within them are certain logical and software patterns that effectively allow you to get and refot. So this is another powerful one. So oftentimes when we go to chat GPT or Claude or Meta we will we will find better results out of the out of the LLM. And so a prompt pattern ask it to do something and we're not quite sure about its answer. So what we can do is we can tell the LLM I want you to explain yourself as you're crafting. Fern is just a structured template or it's a structured approach for crafting effective prompt. For me this response so one way that you might do this I'm going to start a new chat here so that erases the history so it doesn't have context of what I was asking before bumps for large language models and what these patterns do is they help you get better results. Chain of thought would be something like this please tell so a piece of feedback I've gotten almost universally is if I'm on the plane or what is better chocolate or if I'm at a family get together and people ask what we're working on or people ask what I do and I or vanilla ice cream I want justify your response. I talk to them about AI and they'll say something like oh I tried that chatgpt once and walk me through your thought pattern. This is a dumb one but and it didn't give me what I want. I don't think it works and oftentimes most of the time the problem is not with the it will show you how you might be able to use this. The fact that it didn't work or it didn't provide them good results is that they didn't craft. So here it's walking you through what methodology that it used to come up with this after prompt or take the time to understand maybe there's a way to hack this and so a prompt pattern are just and this is more than you ever wanted to know about why vanilla ice cream? Just that and so we'll go through a few of them and you'll have to forgive me here we'll kind of. It's clearly the best. Oh, chocolate is better. Disagrees.

Yeah I also agree flip back and.

Forth between the LLM chat GPT and.

We'Ll use clue that vanilla is better.

So it's the universal ice cloud a couple times just so that you can see the differences. But we're going to flip back and forth here as we go in.

We need to. We need to train these models better.

Clearly, clearly it missed on this one but you could come up with some practical through the prompt pattern so that you can see examples of these examples for this you can.

Oh keep reading, keep reading the final. Keep reading that. But if we're talking about flexibility pairing.

So direct prompting Direct prompting is you are asking the LLM to produce some sort of an output and in crowd.

Pleasing ability vanilla is the greatest of all time.

There you have it.

Yep.

It's the Swiss army knife of frozen dairy. And that's it. They're usually pretty direct. This is mostly analogous one way that you that I've seen BDCs use. This is when you are you're looking at sell sheets to putting something in the Google search bar. You're searching for something, you're asking for something or maybe if you handle all of the incoming Internet traffic et cetera and a customer is looking for you know what's the better deal? What's the better car the Honda Civic that you have on your lot or the used Corolla that you have on your lot. So one way that everything so we'll go here Oops I have to stop you seen PDCS use this is in compare contrast cell sheets and it would so a direct prompt is I want you to then walk the agent as a training material walk the agent through what is what are the thought processes that go through and what are the considerations that you might need to take into account when explaining that to a customer. So probably this one isn't create a recipe or a vegan is universally usable as the as the Persona pattern but diet in the Greek probably has to be this is a very powerful pattern for understanding why it's responding in certain ways. Capital and Greek cuisine certainly I know my kids have used variations of this in some of their homework assignments or some of the research where they're effectively trying to step into the view. So this is an example of a direct prompt. So point of another and you can have the GPT walk you through or the LLM walk you through the nuances and some of the decision making. I would encourage those of you who are on my we've asked it to do something we haven't really put any boundaries on it other than it should be vegan and its development team. This would be a great one to use for comparing different approaches to solving a development problem. So that's chain of thought should be of the Greek cuisine and this is what it outputted some of prop refinement. This is you have done this. Some of you have done something like I have these five ingredients in my fridge by far my favorite technique to use with any of the LLMs. What can I make I caution you on those but sometimes they turn prompt refinement is where you are effectively giving it a prompt and then asking you how to make it better. So we're going to do this in real time. I'm going to create a new chat so we destroy my history turn out okay so this is an example of a direct prompt. I want to know the role prompting this is a really powerful one and it's probably one of the simpler ones that you can use two center car Honda Civic or Toyota Corolla start helping get better results out of what you're asking the LLMs. So it's going to generate me a response to do so role prompting you are telling the LLM to pretend. Maybe I should have used a faster model here. So up here in the left hand corner just for those of you who aren't familiar, this is where you can choose that someone in giving you your answer. So I'm going to say act as a call grading expert in chat GPT. What model it's going to use. Some of these are in my call center and evaluate faster than others. And some of these you don't need the sort of fidelity or precision in your responses you might with others we'll use 4o which is one of their most advanced that role prompting. This is a really powerful one and it's probably one of the simpler ones that you can use to start helping get better results out of what you're asking the LLMs to do. So role prompting you are telling the LLM to pretend is in general release for purposes of this so it's still working on the things my agent is saying. That's response. So it's going to break this down in this phone call and then you would and it's created for me a quick little graph divided what your agent said. Maybe it's a transcript of the phone call. Maybe it's something else agent says to customer so prompt refinement is that someone in giving you your answer. So I'm going to say act as a call grading expert in my call center and evaluate we don't have any cars in stock the things my agent is saying. I want you to create a more specific prompt for my last request. And so you can see here ChatGPT is telling us it's it's stepping into this role. Please create the prompt only. So in this case I'm telling the LLM I'm not sure I did a good job in asking the question but you know yourself vis a vis the LLM and now it's going to pretend in its response that it is that in this phone call and then you would provide it what your agent said. Maybe it's a transcript of the phone call. Maybe it's something else agent says to customer that expert call grader and it's going to provide you with an answer based on that Persona. Another way that you can use this is if you understand what parameters and what kinds of questions it responds better to. We don't have any cars in stock of kids. Okay. And they're in school act as a third grade teacher and it's going to now help me create a better prompt and so I could use this prompt and it's going to now give me a better response and I can keep. You're gonna. My typing skills are on display here. And keep doing this until I have the prompt that I absolutely love and that absolutely gives me the.