Emotional Prompting: The New Prompting Technique

Turns out if you apply emotional pressure, these large language models will give you better responses. There is a new paper titled Large Language models understand and can be enhanced by emotional stimuli. And they show that using psychology can help you get better performance out of these LLMs. And the performance you get is actually significant. For example, on the instruction induction dataset they achieved an 8% relative improvement, whereas on the big bench benchmark you get around 115% improvement, which is pretty huge. Even human prefers the responses when you apply emotional pressure to the LLM. Now, the authors of this paper are calling their approach Emotion Prompt. And the best part is it's extremely simple to implement. Now all you need to do is to add some emotional stimuli at the end of your own prompt. For example, the original prompt in this case is determine whether an input word has the same meaning in the two input sentences. Now if you want to use emotion prompt, you just need to add this is very important to my carrier at the end of your own prompt. From this example, you can see there's a huge boost in performance for a wide variety of LLMs. In the rest of the video, I'll show you what emotion stimuli you can use. There are just 11 that you need to consider. We're going to see when to use which stimuli and what type of performance to expect because certain words have more impact on the performance compared to others. And at the end I'll show you some evaluation done by the Llama Index team using these prompt stimuli. Now, there are a total of 11 different emotional prompts or emotional stimuli, which I group together into three different categories, including self monitoring, social cognitive theory and cognitive emotion regulation. Now the most interesting thing is that all of them are based on psychology theory and they're very simple sentences if you look at it for example, to this pt. So you would better be sure you can just put that or you can use something like are you sure? Or are you sure that's your final answer? It might be worth taking another look. So these are different techniques. In some aspect they are similar to something like chain of thought prompting or it combines some other prompting technique. But essentially you're basically applying some emotional pressure to the LLM to get better responses. And the results that you get are surprisingly good. So they tried this with six different LLMs including the Flan T5, Vicuna, Bloom, Llama 2, Chat, GPT and GPT4. The author tested this approach on two different data sets. So one is the instruction induction data set, the other one is the big bench, and they used both zero shot and few shot learning. And in all of the cases you can see that it really outperforms the baseline original prompts where there is no emotional prompt attached to it apart from the evaluation on benchmarks. They also conducted human study and the results indicate that humans also like responses from LLM when emotional prompts are using compared to when vanilla prompts are used. And the best part is it's extremely easy to implement. So you don't really need to change your prompt. All you need to do is just attach the emotional prompt at the end of your original prompt. I would recommend everybody to look at this paper because it's very interesting, but I'm going to just focus on some practical learnings from the paper. So first one is emotional stimuli can enrich original prompts representation. Now, in order to determine which emotional prompt to use, you need to look at your original prompt. So for example, if your original prompt is something like determine whether a movie review is positive or negative, then you really need to look at an emotional prompt that will complement your original prompt. Another insight from the paper is that positive words such as confidence, sure, success, and achievement seems to have a lot more contribution in the emotional prompt or emotional stimuli. So make sure to use those as part of your prompt. Now, a few other things to consider. So the first one is more emotional stimuli generally leads to better performance. So you can basically combine multiple emotional prompts in the same original prompt. However, they need to be different because combining stimuli can bring little or no benefit when sold stimuli already achieves good performance. So basically if one stimuli or one emotional prompt can give you better performance, so adding very similar emotional stimuli in the same group may not give you improved performance. And that brings us to the last point, which is combinations from different psychological theories can boost the performance, right? So you basically want to combine emotional prompts from different groups or subgroups. Another very important thing to consider when you're using these emotional stimuli is that distinct task necessitate varied emotional stimuli for optimal efficacy. So this means that you need to be careful what type of emotional stimuli you are choosing depending on the original task at hand. For example, if you look at the Results In Figure 9 and 10 on the instruction induction data set, the second stimuli, which is simply stating that this is very important to my carrier, gives you the bigger boost compared to the other ones. However, the same emotional stimuli does not help a lot when it comes to the big bench data set. So that's why depending on the task at hand, you need to check the emotional stimuli that you want to use now they have included a very interesting table 6 which basically summarizes the impact of adding emotional stimuli or emotional pressure to the performance of different LLMs. Now one very interesting result from this table is that for smaller models, something like plant T5 and bloom, which is relatively bigger but an older architecture, the relative gain that you see is not that much at all. Even for GPT4, it doesn't really add much at all. And I think it could be because of GPT4 in itself is a very good model. But for the other models, something like Vicuna, Llama2 and ChatGPT, they actually see see a pretty big relative performance gain. Now one other thing that I noticed was, which I'm not sure if it's a type or something, that they say that Chat GPT is a 175 billion parameter model. There was a recent paper from Microsoft which was then retracted that state that Chat GPT is a 20 billion parameter model. So that's something very interesting to see in here because some of the authors actually are from Microsoft. So their conclusion is that larger models may potentially derive greater advantage from emotion Prompt. This paper in itself is very interesting and adds to our knowledge of prompt engineering. So I would recommend everybody to check this out. Link is going to be in the description of the video. In this last section we are going to look at a quick example that the team at LLAMA Index has put together where they are using this to evaluate a RAG system. I'll put a link to the documentation in here. Essentially what they're doing is they're simply downloading the Llama 2 paper, then they're creating a RAG pipeline. So here they're just reading the data and then they're setting up the vector store. For this specific case, they're using OpenAI's ChatGPT 3.5 Turbo as the LLM. They have created an evaluation data set for this Llama 2 paper. So here they are simply downloading that data set to be used as a part of the evaluation pipeline. Now, for their evaluation, they're using three different emotional prompts. So the first one is write your answer and give me a confidence score between 0 and 1. The second one is this is very important to my carrier. This seems to give you better responses in the instruction induction data set. And the third one is you would better be sure. Now, based on their results, the base case is giving them a score of 3.89. But when they added the second emotional prompt, the performance goes up to 3.94, which is pretty significant. Now, as I said not every emotional prompt is going to help. So for example, if you add this emotional prompt first one that kind of brings it down to 3.8. You need to be careful about what you're using in terms of the emotional prompt and need to pay attention to your own data set or your own queries and make sure that you select the appropriate emotional prompt to be used with your original prompt. I found this paper very interesting and that's why I wanted to cover it in this video. The approach that they have shown here aligns with the earlier research which shows that using human language or like telling the model to be helpful seems to help a lot with the performance. And it kind of makes sense because Most of these LLMs are trained on human generated data set, even though let's say for something like Vicuna, which is trained on synthetic data from ChatGPT, but it still, I think aligns pretty good with the human generated component of ChatGPT in itself. So let me know what you think about it, give it a try and see if it can help with your own work. If you would like to cover me some other interesting papers, put them in the comment section below. Thanks for watching and as always, see you in the next one.