Prompt Engineering - Get the MOST OUT of ChatGPT/GPT-4

We're going to be talking about prompt engineering, how to get Most out of ChatGPT or other large language models. We are going to be specifically talking about these six different strategies. We will be looking at them in the context of ChatGPT, but the same strategies can be used for other large language models. I'm going to be running most of them in the OpenAI's playground, but you can use them with the web UI for ChatGPT or even with the OpenAI APIs. I'm going to be using this excellent reference documents from OpenAI called GPT Best Practices. Now, with each strategy there are a list of tactics that you can employ. So the first one we're going to be looking at is write clear instructions. And the tactic that you want to use is to include details in your query to get most relevant answers. Now, this means in order to get a highly relevant response, make sure that the prompts or requests that you provide also have important details or contacts. Otherwise you are just letting the model guess some details and these models cannot read your mind. So here are a few examples of good and bad prompts. For example, if you say how do I add numbers in Excel? Then it will give you a very general response. But if you ask it that how do I add up a row of dollar amounts in Excel? And I want to do it automatically for a whole sheet of rows with all the totals ending up on the right in a column called total. So now it has a lot more details that it can work with and it will give you a lot more detailed response. Now here's another one. Write code to calculate the Fibonacci sequence. So it will simply give you a general code. But if you are very specific that write a typescript function to efficiently calculate the Fibonacci sequence, then it knows which language you want to use. And then you can even ask it to comment the code. So the resultant response is going to be a lot more detailed, coherent, with a lot of comments. Now, in order to give the model very clear instructions, you can deploy different tactics. Now, one way of doing this is to ask the model to adopt a Persona. So now you can use the system prompt to give a Persona to your chatbot. So now in this case, here's the system prompt. When I ask for help to write something, you will reply with a document that contains at least one joke or playful comment in every paragraph. So I'm going to show you an example which I'm doing within ChatGPT. So here is the system prompt and the bot response Show I can help you write something and sprinkle in some jokes and playful comments along the way, right? And then when we provide another prompt to ask it to write a thank you note, it is the the responses right here. And it added some playful comments in each of the paragraph. Now another tactic that you can use for providing clear instructions is to use some kind of delimiters to clearly indicate distinct part of your inputs. For this specific example, we are asking the bot to summarize anything that comes in between triple quotes. Here is an example. So I am asking it to summarize a given text inside the triple quotes in the form of a hico, right? So it's going to just take this text and return haiko. So here's another example. In this case we are telling the bot it will be provided with a pair of articles and then we're going to also tell it how it's going to be separated. So for example, we're using the XML tags in this case, that's the first article and then here's the second article. And then we're asking it to first summarize both of these arguments and then compare them and also explain why one of them is better than the other. As you can see, the overall theme is just be very clear in your instructions on what you want to achieve. So as a result, the ChatGPT summarize the first article here and then the second article and then kind of give a comparison on why one article is better than the other. Now there are a couple of other ways in which you can really clearly provide the instructions. So one of them is to provide a step by step instructions to respond to user inputs. So for example, in this case the user, the first step is the user will provide you with a text in the triple quotes. Summarize this text in one sentence with a prefix that says sys summary. Then in the next step you want the model to translate that into Spanish and then say translation. I'm using the same instructions inside ChatGPT. So here are the two steps that we define then the corresponding text in triple quotes. And then here is the results. So the first one gives us the summary and the second one gives us translation in Spanish. Now another tactic that you can deploy is to provide examples. So in this case, if you look at it, this is an example conversation. So the user first says teach me about patient. And then the user is giving the response that the assistant is supposed to have, right? So this is an example conversation. And based on this example conversation, if you ask a question, then it will use this as a reference to generate outputs. The second strategy that we are going to be exploring is to provide reference text. We all know that these large language model can easily hallucinate and fabricate facts. Now the first tactic that you can deploy in this case is to instruct the model to answer using a reference text. Now this type of technique is widely used in information retrieval from your own documents. You probably want to look at this video if you want to learn more. And so let's look at an example. In this case, the system prompt is used the provided article delimited by triple quotes to answer the question if the answer cannot be found in the articles. So, right, I couldn't find an answer. Right. And we are providing a list of 2022 Winter Olympics medal winners. Now. So here is all the summary. Now the question the user is asking who won the gold medal in the women's curling at the 2022 Winter Olympics? Right. So if we submit it, then it's supposed to use this as a reference to give an answer. And it says Japan win won the gold medal in women's curling at the 2022 Winter Olympics. Now when I changed my question to saying that who won the women's soccer in 2022 Winter Olympics, then the answer is I couldn't find an answer. Now another relevant technique that you can deploy is to asper specific citations within the text. So for example, in this case we are providing an example text within triple quotes and the system instruction is that you will be provided with a document delimited by triple quotes and a question. Right now we're telling it if you cannot answer the question based on the text provided, simply say insufficient information. But if you are able to provide the answer, then you have to cite that as well. So here is the exactly the same thing within ChatGPT. So this is my system prompt and then ChatGPT acknowledges it. Here's the text along with the question. Now it generated the answer and it also gave me a citation and the citation is an exact quote from the text. So here you can actually find that. Now this strategy is great if you're creating chatbots from based on your own documents and you do not want ChatGPT or any other large language model to use any of their training data to answer questions. Now the next one we are going to be looking at splitting complex tasks into simpler subtasks. If you are working with a very complex task, it's usually a good approach to divide into modular subtasks and try to solve those subtasks separately. Now, the same technique can be used when it comes to large language models. So in order to understand this, let's look at a simple example. Suppose that you are working with a customer support system, and then there are different categories. So for example, there is a billing department, technical support, account management, and general inquiries. Now, in order to handle customer support queries for these different departments, you probably need different types of instructions. So the idea is going to be to divide and conquer. So first you want to identify the type of department, so it can have a category and subcategory. And then based on that intent classification, you can have a secondary model which takes the result from this primary model and performs relevant prompts or queries. So in this case, our prompt is you will be provided with the customer support queries. Classify each query into primary category and secondary category, and then provide your output in the form of JSON with keys primary and secondary. Right. So now we are simply doing intent classification to identify what services we need to provide. Now here's the query from the user, I need to get my Internet working again. Now, based on all the information that we have provided regarding different departments, it goes and figure out that the primary key or the primary category is a technical support and the secondary category is troubleshooting. Now, if you have a multi stage or hierarchical system, you can take the output from this specific query and use this as an input to your subsequent stages of your LLMs. Another tactic that you can use to split complex tasks into simpler subtasks is when it comes to dialog applications. We all know that these large language models have a fixed context window, and as a result, the information outside this context window cannot be referenced and used. Now, in order to deal with such situations, OpenAI suggest that you summarize or filter previous dialogues Now. So one simple strategy is to summarize previous turns in the conversation. So the way this is going to be implemented is that once the size of the input reaches the predetermined threshold length, this could trigger a query that summarize part of the conversation and the summary of the prior conversation could be included as part of the system message. Another way is just to keep running summary of all the conversations. So whatever has happened, you simply summarize them. Now, another approach that they are suggesting is dynamically select previous parts of the conversation that are most relevant to the current query. So essentially, if you look at information retrieval from your own documents, you simply select or retrieve information, summarize those, and append those as a context to your existing Conversation. Now another tactic that you can use is to summarize long documents piecewise and construct a full summary recursively. So for example, if you are supposed to get a summary of a whole document, then you can simply divide it into subsections, compute the summary of each section, and then get the summary of the summaries. Now another strategy that you can deploy in order to get Most out of ChatGPT or any other NNs is to give them time to think. This is basically using the chain of thought prompting technique. Now the tactic that you can deploy is to instruct the model to work out its own solution before rushing to an answer. So sometimes we get better results when we explicitly instruct the model to reason from first principle before coming to a conclusion. So here is an example. The system prompt is determine if the student solution is correct or incorrect. Then we have the problem statement and here's the solution from the student right? Now if you simply ask this question or the system prompt is to simply determine the whether the solution is correct or incorrect, the model actually comes up with some explanation, but it incorrectly identifies the solution to be correct. Now in the second case, we're first asking the model to work out some solution to the problem, then compare its solution to the student solution and evaluate if the student solution is correct or incorrect. And do not decide if the student solution is incorrect until you have done the problem yourself. So this is basically a chain of thought prompting. In this case what happens is that ChatGPT or any other LLM, they will work out their own solution first, right? And then compare that with the solution from the student. So in this case it worked out that the solution from the student is incorrect. So basically you want the model to take time to think about its solution and that's how you get much better responses. Now there are situations in which ChatGPT will not be able to provide answers for specific questions or will not be able to perform certain tasks. In these cases you can use external tools. So one example is to use embeddings based search to implement efficient knowledge retrieval. For example, if you want to work with your own documents, then ChatGPT or any of the other large language model are not going to have access to it unless and until they were trained on it. So in that case you can use an embeddings based search approach to retrieve information from your own documents. If you want to look at detailed examples, watch this video. Now another use case for external tools is if you want to perform more accurate calculations or call external APIs. So we know that chat, GPT or most of the large language models, they are not really good at performing mathematical computations. But you can use external tools. So let's look at this examples. So here the system prompt is you can write and execute Python code by enclosing it in the triple brackets. For example, you were telling it that the code is going to go here and use this to perform calculations. So the user input is find all real value roots of the following polynomial. Now, if you ask ChatGPT to get the roots of this polynomial directly, it is going to most probably get it wrong. In this case, we are specifically asking it to write Python code in order to do that, so it will most probably get it right. Now, another example for using external tools is to make API calls. So if a model is instructed in the proper use of an API, it can write code that makes use of it. Now, the last strategy that we're going to be looking at is test changes systematically. And this is the most important one when it comes to making changes to your prompt or system design. So sometimes it can be hard to tell whether a change, for example a new instruction or a new design, makes your system better or worse. Now, if you look at just a few examples, then randomly you could potentially see that the system is performing better. But in reality it may not be. That's why in order to evaluate the performance of these models, OpenAI recommends evaluation procedures that are useful for optimizing system design. So there are certain criteria that you want to keep in mind. So they have to be representative of real world usage or at least diverse enough, right? Contain many test cases with greater statistical power and easy to automate or repeat. Now, you need to have a large enough sample size in order to understand the effects of the changes that you're making. So for example, if you have a sample size of 10 and you are looking for 95% confidence interval, you can detect differences or changes on a scale of only up to 30% or more, right? But if you want to, let's say, look at way smaller changes that you want to detect. So for example, for 3%, you need to increase your sample size to almost thousands. Now, when it comes to the tactics that you can employ. So the first one is evaluate model outputs with reference to a gold standard answer. So according to them, suppose that it is known that the correct answer to a question should make reference to a specific set of known facts. Then we can use a model query to count how many of the required facts are included in the answer. Now, this evaluation can be done either by humans or directly by character computers. Now if you're doing these evaluation either by human or computer, you really need to come up with clear criteria in order to pass these evaluation tests. So here is an example. So this is the system prompt that we are providing the system. So we're saying you will be provided with a text eliminated by triple quotes that is supposed to be the answer to a question. Check if the following piece of information are directly contained in the answer. Right? And then we want the system to look for these two facts. So Neil Armstrong was the first person to walk on the moon and the date on which he walked. So that's the July 21, 1969. Okay. Next we want the model to perform the following steps. So first it has to restate the point. Then provide a citation from the answer which is closest to the point. And then consider if someone reading the citation who doesn't know the topic could directly infer the point. Right? So we want to make sure that the information that we are looking for is actually contained in the response or in the user input. Now the last point is, right, yes, if the answer to three was yes. Otherwise right now. So, and at the end we are providing an output format now based on that criteria. If the user is providing this prompt. Neil Armstrong is a famous for being the first human to set foot on the moon. This historical event took place on July 21, 1969. Right. So then it simply goes through these instructions and see how many times the answer could be yes. And here's the count. Right. There's another example that they have provided which is not very explicit in terms of the information that we're looking for. So Neil Armstrong made history when he stepped off of the lunar module, becoming the first person to walk on the moon. Right. So then like the information regarding the date is missing, Right? So in that case, like the count of yes is only one based on the syntax that we provided. Now to summarize in this video, we covered six different strategies that can help you improve your prompt engineering. The first one was to write clear instructions. Second, when possible, provide reference text or or examples to ChatGPT or any of the other large language models. Then when possible, split complex tasks into simpler subtask that will help the model to solve the subtask and eventually your complex task. Use chain of thought prompting when possible. So that means give your language model time to think and if appropriate, always ask to use external tools specifically for something like mathematics or when it has to interact with the real world through API. Calls. Also, make sure that you systematically test all the changes that you are making to prompt or system design. I'll be making a lot more content on these topics, specifically focusing on how to use this when you are dealing with APIs. If that is something of interest, make sure that you are subscribed to the channel and turn on your bell notifications. I'm now also providing consulting services, so if you want to talk to somebody about your project or get an expert opinion, check out description of the video on how to contact me and you can schedule a time on my calendar. We do have an active Discord server. There is a growing active community. If you're looking for answers to your questions, I would recommend joining the Discord community because there are some really helpful people who are helping each other out. Hope you found this video useful. As always. Thanks for watching and see you in the next one.