Prompt Engineering with GPT & LangChain | Get Better Outputs Instantly

Welcome to prompt engineering with GPT and LangChain. LangChain is a very popular open source framework that helps us add extra functionalities when integrating generative AI capabilities with applications or data platforms. In this module, we will get a gentle introduction to the framework and some prompt engineering techniques, and understand some of the basic concepts of LangChain. In later modules, we're going to look at some of the more advanced concepts such as retrieval, augmented generation. So in this module, we're going to start off with calling the OpenAI models. Through the framework, we're going to set up some prompt templates that will write reusable and dynamic prompts. These can then be versioned along with the rest of our code base. We're going to work with LLM chains, which is basically the combination of large language models and these aforementioned prompt templates. Then we're going to look at the output parser in the LangChain framework, something very valuable that can instantly generate certain objects like Python lists, dictionaries, JSON objects, et cetera, that we can then instantly use further in our code. Next, we're going to work with LangChain agents and tools. These will allow us to do many cool things like instantly executing code that's being generated by the model, or let's say, do SQL lookups, etc. And then lastly, we're going to look at the OpenAI moderation API. This is especially useful when you're trying to, let's say, build your own API. No, build your own AI application. You want to have something like this as a filter in between the user and your actual model endpoint. Now, for this project, we're going to use two small text samples. I wouldn't necessarily call them data sets. It's basically just five to six line samples. They're being kept very short so that every time we print our results, we can just get a very simple overview of everything. Just keep in mind that all the techniques that are taught here and all the code that is used here is very scalable. We can just use it on any size of data set. Now, in the file system here, let's take a very quick look at these headlines. So we have six financial headlines on Finnish companies and whether their profits rose or decreased or other information, such as company X has been awarded a contract for a certain project, etc. Now, the goal is to do some financial sentiment analysis here. So, for example, using these GPT models, we automatically want to generate investment advice, for example, based on the current news. So let's go back to our Notebook. Oh yeah, maybe a good thing to mention if you ever get stuck, there is also this notebook solution here. So you will find the solution to all the code problems in this solution notebook. Before we get started, one of the prerequisites is that we need to an OpenAI API key. Now for this OpenAI API key, this has been covered in the previous module, how to get one and how to add it to the environment variables. If you don't know how to do this or you have not followed the previous module, you can always check out the getting started notebook in the file system. With that being said, let's start with our setup. The first thing that we want to do is do some installs. So we're just going to install some packages. You might get some warnings or sometimes even some some errors with these with these versions. But just for this project, these are not breaking changes. So don't be afraid. You can just continue the project with any with the versions mentioned here. Now with those being done, let's start with our first instructions. The first thing that we have to do is some basic imports to basically fetch our environment variable and set it within the OpenAI package. So let's first import the OS package and that's being done by just simple import OS statement. And we're going to do the same thing for the OpenAI package, just import OpenAI. Then we can set our API key. So OpenAI apikey and we want it to be our environment variable that we can call with OS nvidim. So I want to have the OpenAI API key. Just going to run this cell to set our key and we should be good to go. Next we are going to start just doing some basic calls to the OpenAI models from LangChain and that's being done using the OpenAI and the chat OpenAI classes. The OpenAI classes are used for completion models, whereas the chat OpenAI are used for the conversational models. So maybe a quick overview here. Completion models refers to typically the older generation of models, if we don't count the new instruct models. But basically they just give like a single completion to a text input, whereas the chat models are made for conversational use. So the models used in ChatGPT, which are GPT 3.5 Turbo or GPT 4 are the chat conversational models. So they will be loaded using the chat OpenAI package. No chatopenai class. So sorry, but first let's start importing these classes. So these are also just very straightforward imports. So from LangChain LLMs we want to import OpenAI. And from LangChain Chatmodels, we want to import chat OpenAI. Okay, run the cell. We have done all of our imports. Now let's get started on task one. We want to import our financial news headlines data. So we have taken a look at the sample text that we have. The financial headlines we just want to read in this text file and basically store it within a simple python list, let's say. So what we're going to do is we're going to open the text file in a read format, we're going to use the readlines method and basically assign it to a new variable called headlines. So let's do with open then the name of our text file, which is basically financial headlines. Oh, let's not do that, which is basically financial headlines. Txt. And we want to open this in a read format. Let's open this as, let's say data. And then we can read this in by doing headlines equals data readlines. To get a quick overview, let's also print out this Python list. So here we basically have our headlines loaded into a list, just a list of strings with every element being a single headline. Now we might notice that we have this new line character at the end of every line. Let's just get rid of doesn't really influence the large language model that much, but just for like sanity, let's get rid of these characters. To get rid of these characters, we can simply use the strip method for the first string objects within Python. So to make it a little bit easier, I'm going to use a list comprehension here. So for headlines, let's do x4x in headlines. So I'm just going to loop through headlines and instantly generate a new list and reassign this to headlines. Now the operation that I want to perform here is basically applying that dot strip method to every element of the headlines. Strip basically removes any character at the end of the string and I want to remove the n. Now let's print out headlines again and look at our result. So now we basically have a bit of a cleaned up version of our headlines without these new line characters everywhere. For the second task, we're going to learn how to set up prompt templates. Now, prompt templates, as I've told you before, they're made to be like reusable and dynamic prompts that we can version within our database, within our code base. So basically we can save these with the rest of our code. We can add it to our repositories and basically reuse it in any Way like this, we can add input variables to these prompt templates to make them dynamic. And within LangChain, we have built in verification tools that kind of check whether that input is always given when it's being called from anywhere else, which makes our code more robust. Now, the format of these prompt templates, they are kind of structured like F strings in Python, basically meaning we have a typical string, and if we want to add any dynamic variable within the string, we're using the curly brackets here. So, for example, if I want to have a prompt template to give me suggestions for a trip to any kind of city, I could set up a quick template that says, can you give me some suggestions for my trip to. And then a city variable. This allows for dynamic templating, basically, or dynamic prompting. Now, we have two different types of prompt templates, and that has to do with like the two different types of models that we've spoken about earlier. Basically we have the prompt template and the chat prompt template. One is for the completion models, one is for the chat models. The reason for that is because the completion models, they basically just generate once it's like single input, single output, while chat completion, they can store prompt histories, etc. And these templates will also be able to store certain prompt histories and a system message as well. So the system message, you might have seen this in the previous module, but this is some sort of meta prompt that we can use to do all sorts of things, such as putting the model into a certain role, asking it to. Use a certain tone of voice, or specify specific output formats, etc. Now, to create a prompt template, we are going to use the prompt template class in LangChain. So for a very simple one, we're just going to use the from template method in the framework, and then we basically have to pass along the template itself, the text itself, the prompt itself. So for that, we're just going to use this prompt, and we can already see that we have one input variable here. So we have headline, which is basically a dynamic variable within our prompt. So we can use the same prompt template against our entire data set of headlines. Then we're going to format this prompt, which is basically passing along a certain headline for that input variable, and we're going to see what the end result looks like. So first let's start by importing the prompt template class, and that is done by from LangChain prompts, we want to import the prompt template class. Next we want to basically set up a new prompt template, and we're just going to Call it, we're going to assign it to a variable called prompt template in Snake case. So we're going to do this by prompt template from template and in here. Basically we just want to use our prompt that we have here. So that's that. Now we will see more advanced ways of, of setting up these templates with some verification methods further down the line. But for now, let's just create a very simple example of a prompt template. Now, let's say we want to format this one with the first headline of our data set. What we're going to do is we are. First of all, we're going to assign our result to a new variable called formatted prompt and we're going to call this dot format method on our newly created prompt template. So I'm just going to do prompt template format and then it will expect this input here. So this headline input. So let's say headline equals. I'm going to take the first element from our headlines data set. So remember that headlines was our Python list with all of our headlines stored in there as a string. I'm just going to take the first element there by putting in index zero. And let's take a look at our formatted prompt and what this looks like. So here we see basically our instruction. So analyze the following financial headline for sentiment and then combined with the headline that we fetched from our data set. Now that we have set up a prompt template, let's continue towards a chat prompt template. So here we can have some prompt history if we want, but mainly we get access to this system message that's available within the GPT4 model and within the GPT 3.5 turbo model. So the models used in ChatGPT first, what we want to do here is import the other class. So chat prompt template from the same package and we're going to use this system message. Now, the system message, if we're going to look at how it's written, so you might remember I told you, like system message can be used to put the. To put the model into a certain role. And that's what I'm doing here with this first sentence. So I'm telling the model, what are you doing right now? Or what kind of role do you perform? So I'm telling you are performing sentiment analysis on news headlines. Regarding financial analysis, the sentiment is to be used for blah, blah, blah, and the format of the output has to be consistent. So the thing is, I want the GPT model to only output either the words positive, negative or neutral and nothing else. I don't want it to respond to me with like two sentences telling me whether it's like slightly positive or kind of negative. I just want to have the single word positive, negative, neutral that I can then later on use for analytics. So we're going to use this system message and we will. Yeah, we will start by instantiating a new chat prompt template, just as we did with the other template. And instead of from template, we're going to do from messages, we're going to add a system message and a human message. So a human message is basically like the user prompt. This will be put into a tuple together with their respective contents. Now we're going to format that template in pretty much the same way as we did with the previous prompt template and just check out the results. So first of all, let's start by importing our prompt template. So from LangChain prompts we want to import our chat prompt template. Next we want to, we want to define our system message. So we will just copy this string up here and assign it to the system message variable like this. Next we are going to instantiate our chat prompt template. And the name for this variable is chat template. So let's call it chat template. We're going to set up a new chat prompt template from messages. Now, this will take in a list of different messages. So let's start up with a list and we are going to do a small list of a single system message and then a human message. But if you want to, in these chat prompt templates, you can also add like a bit of conversation that already happened before you are using this certain prompt. So you could add like a human message, an AI generated message, and then a human message as well to simulate like the start of a conversation already. Now, for us, we're basically using this just to get access to a system message. So what we're going to do is we're going to pass a tuple and then first of all we're going to say this is the system message. So here we can do system or human or stuff like this. So let's start with system message. And then the second element of our tuple is the content itself. So we've just stored the system message in the variable called system message. So I'm just going to call that one here. That's basically this long string here that we just defined second, going to make a second tuple, and that will be our human message. And for our human message, we basically just want to have like this same, the same one that we used here within our prompt template. So this will become our human message. Now, again, you can see that we have one single input here. So again, we have this headline input that we want to format our prompt template on. So let's quickly go back to which one. Yeah, so we want to have it assigned to formatted chat template. So we're going to define formatted chat template equals our chattemplate format. And again, just for that headline variable, we want to pass along our first headline. Now let's see what this looks like once we start to print it. So this is a bit more elaborate than the one before because here we first have this entire system message that we defined and then we have our human message. This is basically what the whole ChatGPT application looks like as well, behind the scenes. So behind the scenes you also have the system message integrated in there and your input prompts in ChatGPT will basically be defined as this human message. So now we have set up a prompt template and a chat prompt template. Next up, we're going to set up some basic LLM chains. Now, we're not going to go too deep into this, but the concept is quite simple. An LLM chain is basically the combination of a model and the templates that we just have set up. And then we're able to chain multiple of these objects to each other by basically adding the output of a certain LLM chain as the input for a next LLM chain. Now, again, we have this for completion models and chat completion models. So we have two different chains. But first of all, let's start with, with using it for our completion models. So here we're also going to do the necessary imports. So that is from LangChain chains we want to import LLM chain. Now first of all, yeah, we're going to reuse this OpenAI class that we imported all the way at start and basically calling or instantiating one of these OpenAI classes will give us access to a model, to a GPT model. Now, the OpenAI instead of the chat OpenAI, as you might remember, by default that will, or in the current version, at least for LangChain, that will steer us towards the Text DaVinci model. And we can instantiate multiple of these classes to, for example, change parameters. So I could instantiate one with a temperature set to zero or a higher temperature and maybe to touch a little bit on that one, a very simple and brief explanation of the temperature is basically the randomness that we put into the model. Like the higher the temperature, the more random a model Becomes which is favorable for creative use cases. While if we have like use cases that require a lot of precision, like we want to have more factually correct answers, or for example, when generating code, we generally prefer very low or even zero temperature to get the best result possible. Okay, so let's start by instantiating one of these LLM chain classes. So we're going to assign it to a variable called completion chain. So let's create that one first. Completion chain equals LLM chain. And here we want to have two different parameters. So we have LLM and we have prompt. Now for LLM, let's just quickly instantiate a new class, instantiate a new model object here, just a simple one, just using the default parameters here. And for the prompt, let's use our prompt template that we've defined earlier. So that is our prompt template. Now remember, this is not our formatted prompt. So this is where we have this flexibility. Like every time we run this chain, we can run it with different inputs. So again, let's try to run it on our first headline. So we do our completion chain run and as input for that headline, for that headline input that we had in our chat prompt template, we're just going to pass along our first headline. Now we're going to automatically format our prompt template and send over that prompt template to a completion model. So let's see what we get as a result. We get N N positive. So. It classified the first headline as positive, but it still has these characters in front of it, which we don't necessarily want, but we will get rid of them later in this module. Next, let's do the same thing, but using our chat prompt template. So for this we're going to be creating an LLM chain, but just using the chat OpenAI model instead of the normal OpenAI model. The concept is pretty much the same. So let's say chatchain also equals llmchain. And again, we have these same two parameters. So we have LLM and we have our prompt. Now, Instead of the OpenAI model, let's create ChatOpenAI model by default. This will direct us to the GPT 3.5 turbo model, but we could also fill in, for example, GPT 4 or anything like this. And for our prompt template, we have our chat template that we've created earlier. Now again, keep in mind that this is not the formatted template, so we have that flexibility to format it while we or when we run the chain. So for that I'm just going to do chat chain.run and with this template we had two or we have two input variables because we also now have that system message that we can tune. So for the headline I'm just going to fetch our first headlines again like this, and then our system message, because we can change this with every call to the model, we have stored this in a variable called system message as well. If we run this, we get positive. So both the completion model and the child completion model classify that first headline as positive, which is a good thing because if I remember correctly, there was a headline that said like that company's profits went up from X to Y. So for task four, we're going to make use of the output parser in LangChain. The output parser, as said before, is very useful when we want to have like Python objects straight away from our large language model output. Now it's good to know that the large language model itself, it will always just output strings, it will always just output texts, but the output parser will has some built in tricks to automatically create like Python lists, date, times, jsons, dictionaries, etc. Now how it does that is first of all it will parse the text output and basically create that specific Python object from it. But on the other hand, it also adds format instructions to our prompt behind the scenes. So it will actually add some extra text to our prompt to make that the text output of our large language model is in the necessary format that we asked for. Now in this specific case, we're going to extract the company name from every headline. Later we can then basically set up a data frame that combines the company name with their respective sentiment to basically have a very basic version of financial news analysis. So what we want to do in terms of output parsing is when we pass along all of our headlines, we want to get a Python list which basically contains all of the company names. Now to do this, first we're going to make the necessary imports. So we're going to be using the comma separated list output parser and we're going to fetch the format instructions from it. This is basically what we're adding behind the scenes to our prompt. So we'll have to pass this along within our new prompt template as well. So with that being said, we're going to set up a new prompt template. We're not going to use this from template method because we want some extra flexibility here. And this will allow us to basically call the init function of this prompt template, which has or where we're going to use three parameters. First, template, this is Basically what we've used before. So this is where we actually give our text prompt as a string. And here we're going to ask it to list all the company names from the following headlines. Now, one of the problems I've noticed while developing this notebook is there is one headline, I think with two company names in there. So we just want to limit it to one company name per headline for now, just to have like the easy analysis so we can map them one to one to the sentiments that we will get later on. And you can see that we will insert all the headlines this time, not just a single headline. So we're just asking it to list all the headlines, to list all the company names from all these headlines. And then afterwards it will automatically add like these format instructions like this. We're going to specifically define an extra parameter that's input variables. And we do this to verify that these input parameters are actually being inputted. Like if we, if we try to call this prompt template without inputting certain headlines, then we will get an error. So that is just a best practice to get robust code, especially when using this in, like, larger applications, then there's partial variables. This is basically where we, where we pass along the format instructions that we get from our output parser to have it parsed in a list. And then we're going to look at the result. Oh no, we're not going to look at the result. We're just going to format the prompt template in this case. And then the next one we're going to look at the result because we still need to run it through the model. Like with this code block, we're just going to end up with a formatted prompt template that includes output parsing. So first let's start with our necessary imports. So we need this comma separated list output parser from LangChain outputparsers. So from LangChain outputparsers, import comma separated list output parser. Then we're going to instantiate a new one and we're going to assign it to a variable called outputparser. So I'll just use outputparser equals comma separated list outputparser. And we don't need to pass along any parameters here. The default ones are more than fine. Next we're going to fetch those format instructions from that specific output parser. And that is very easy. That's just calling this method the GetFormat instructions. And let's just store these under format instructions so we get them from that comma separated list output parser. By using outputParser getformatinstructions. Now we are going to set up our new prompt template to get the company names from all of our headlines. So the name for the variable that we're going to be using here, I see that it's not listed here, but let's just use company name template. So this will become a new prompt template, and we're just using the INIT method. So we're just doing it like this. And here we will have three different parameters. So we have template, we have input variables, and we have partial variables. For the template. We have it here. So we're just going to copy this one and add this string in here for our input variables, we basically pass along a list with our input variables as a string. So keep in mind that these format instructions, they do not count as an input variable. They are partial variables. So the only input variable that we have here is headlines and then partial variables. That's basically these format instructions. And here the code is expecting a dictionary. So we want to have a key value pair that says format instructions and that equals the format instructions that we've received here from our output parser. So to get a quick look at our template, let's just format that template with our headlines. So let me see here. That is basically we're going to call it formatted Company template to keep the same structure as before. And we want to basically do company name template dot format, much in the same way as we've done before. And now as an input variable, we have these headlines. Now what we want to assign here is basically that string, that list of strings that we had before that we read from the text file, and that is also the same name. So headlines, the headlines variable. Now let's print this and yeah, maybe let's just print this as well to have a look. So basically what we see here is our instruction plus our list with all of our headlines, but within a. Within a list that is being converted to a string. And then what we see at the end here is basically those format instructions that we have added. Those format instructions are then basically being, or they put it in the right format to be parsed by the LangChain framework. So next what we're going to do is we're basically just going to run this template through a model. But now instead of using an LLM chain, let's try a little bit of different approach. So you have like multiple, so, you know, multiple approaches to actually make use of models and make use of templates. So in this case we're just going to set up a OpenAI model. So the OpenAI model here, and we want to set its temperature to zero. The reason for that being we want to have it as precise as possible. We don't need randomness at this point. We want to be as precise as possible on extracting these company names from our headlines. So the way we do this is we're basically just going to create this OpenAI or create an instance of this OpenAI object again by default by referring to the text DaVinci model. And we're going to pass temperature equals zero. So we have a model with zero temperature. Now we're going to run that model basically by just calling that model variable and we're going to store our output in underscore output. Now, an underscore, basically it indicates that we're just using a temporary variable here and we will override it a couple of times as well because we just want to have like this intermediary step in between. Now you could chain all of these methods, but for this module, just to keep it a bit more easy to follow, I'm adding this extra step here. So for model, to run this model, I'm just going to do model and ask and insert our template here. So that was our formatted company name template. Next we want to parse the output. So we will need our output parser. And that output parser has this parse method with which we can parse that output that we got from the model. We're going to assign this to a new list that we've created, that is company names. So this will instantly be a python list because of this output parser within LangChain. That's the whole power of it. So our output parser parse and we parse the output that we've received from the model. Then let's quickly print the data type of this company names variable, just to show you that we indeed have a python list object straight away. So that is done by doing type, then our variable itself, which is company names. And lastly, I'm also just going to print the company names themselves so that we can have a look at them. You might see this takes slightly longer because we're adding more and more functionalities and later that might even increase. But here we see our class is indeed a list. So we instantly have a python list here that we can use like further down our code and we've extracted the company names from our headlines. Now, with that being said, let's move on now to agents and tools to Continue with these company names. Next up, we are going to work with agents and tools in LangChain. Now, agents and tools, an agent is basically, let's say, a bit of a code executor that can work with the, the output of an LLM and can also call different tools if we want to. So an agent is kind of like the intermediary step or the intermediary component that glues all of our components together. If we have like user or our web app front end on one end, our GPT model on another, and like external tools, etc. The main component in the middle that glues everything together is our agent. Now, to really understand some of the capabilities of the agents, we first want to touch upon the concept of Merkle prompts. So Merkle prompts are modular reasoning, knowledge and language prompts, and they come in different formats. But a format that's very popular. Is basically looping through a certain set of steps. So we ask a question in the very first loop, this is just going to be the user question. And we tell the model explicitly to think about what you should do as a next step, like explicitly adding this, like ordering it to think about what to do actually increases the reasoning capabilities of the model. Next, it will perform an action. And the definition of an action here is basically picking a tool from a list of tools that we give it access to. So we can define, let's say a certain toolkit. Let's say one tool is a Python code executor. Another tool might be an SQL database, another tool might be our calendar to look up availabilities. Another tool might be a calculator, etc. So it will pick a certain tool from our tool list with a certain input. So that's the action input. So for example, when we are using a calculator, that action input will be, will be the actual equation. When we're doing an SQL lookup, that action input will be an SQL query, etc. Then the observation is basically just the result from step three and four. And then with that observation we're going to think, is this the final answer that we expect from our question? If no, then we're just going to loop through, loop through and continue this until we get that final answer. So what are tools and agents? I've already kind of touched upon this, so I will leave this to you. Next up, let's start by actually creating one of these agents. So before we actually want to create an agent that's going to assist us in our whole financial sentiment analysis, we are going to run a very little quick test. For this test, we're just going to ask the GPT model to perform a, or to solve a mathematical equation. Large language models are typically not that good at performing mathematics. Like they are taught how to generate language and not necessarily equations. Like, okay, they might know some equations, but that's because they have learned those from their, from their training data set. But if we're going to replace certain numbers by very random numbers, then it will get lost because it does not know the actual reasoning that it has to do. It might know that, for example, two plus two is four, but it does not know that X plus Y equals the sum of x and Y. So to say. So to solve this, we're just going to create an agent and we're going to give it access to a Python tool. This will allow it to instantly execute the Python code that GPT model generates. Now we're going to create this agent and we will need three parameters. So the LLM, the tool, and I've also set verbose to true here so that we can actually see what's going on in these intermediary steps. Then we're going to run that agent on a mathematical equation. Now, I have been looking for a tool that requires the least amount of setup from your side. So that's why I've not chosen to do like SQL databases or something like this, because then again, you would need to go somewhere, set up more API keys, etc. So we're just going with a tool that can be used very, very fast. Just keep in mind that if you want to use other tools, it pretty much works in the same way. So let's get started. First, we're going to do the necessary imports again, and that is from LangChain Agent Agent toolkits, we want to import our Create Python Agent function. Oh, this has to be agents, sorry. Then from LangChain Tools, not Python Tool, we import Python repl tool. Cool. So now we can start creating our agent and we're going to assign this to a variable called Agent executor. So Agent executor equals Create Python agent. And we need three parameters here stated above. So we need LLM, we need tool, and we set verbose to true just so we can see all the output. So for the LLM, we could be reusing some of the previous OpenAI instances that we've created, but we want to change some, we want to change one of the parameters a little bit. So let's just create a new one here. First of all, we want to set temperature to zero because we want it to be as precise as possible. And next, a good practice when using these agents is also specifically configuring our max tokens because we want to, Sorry, we want to limit the size of our output, since these tools or agents can perform certain loops and we don't want to run the risk of having a massive output that could skyrocket our prediction costs. So just safety measure here. When we're working with agents, we're going to configure some max tokens. Then finally, as a tool, we're just going to create an instance of this Python repl tool that we have imported. So we're going to ask the agent to run a certain mathematical equation, which is basically that agent, agent executor run. And we want to have this string here. So we're going to ask it, what is a square root of 250? And round it down to 4db. We're not using a calculator tool, we're using the Python tool. So we expected to generate some Python code and instantly execute it. So I was expected to use the square root function and then the round function on that result. It will probably chain it, but let's see what happens. So the intermediary steps should be showing up in green normally. And here we see like the structure of these Merkle prompts. So this is basically our thought. Like we asked it, like, think about what you need to do. It says, I need to calculate the square root of 250. Okay, sure. And it picks an action. Now, for now, we have only defined one tool, so there's not much to choose from, but it indeed chose for the Python repl tool and as input, that will basically input the Python code that the large language model generated. Now the observation is basically the output of that whole task, so that is equal to this. And then we're going to go into our loop again of our Merkle prompt until that thought says that we have the final answer. Now, in this case we only needed one step because it now knows like, okay, I have the final answer. So it's going to return the final answer and finish that whole chain. And here we have it, the answer. Okay, now let's try to use that agent to help us with our financial news analysis. For our financial news analysis, we're going to ask our agent to do a bit of an elaborate task that requires a bit of code as well. So first of all, we want to have the company name from every headline and we Also want to have that financial sentiment. So we want to know is that headline positive, neutral or negative? Then we don't just want to have these two lists, but let's say as a end result, we instantly want to have a CSV saved locally with all of that information in there. So we're going to ask it load this data into a PANDAS data frame. I'm quickly describing here what the schema of the PANDAS array frame will look like. So I want to have three columns. First one is the name of the company, the second one is that financial sentiment. And then lastly I want to have like the raw data, the headline itself in the, in the third column. Then I want to just save this data frame to a CSV file and I pass along a file name that I wanted to use. And just to be safe, because I know how like the Pandas to CSV method works, I want to have that override is true argument set to true. So I just say if that file already exists, make sure it should be overridden, then I'm just going to pass along the headlines itself. Now normally we would be doing this with a prompt template as well, if you want to keep this modular, etc. But we've learned how that works and now we just want to go fast through the other stuff. Otherwise we just keep creating more and more and more of these prompt templates every time. But just so you know, in scalable solutions in large applications, this is typically what you would set a prompt template for. So what we're going to do is we're just going to run that agent so that in terms of code, very simple, we're just going to do the agent executor run again. I'm going to copy this and basically paste that here and let's see what happens when we run this. So basically. It gets all of our headlines. It creates two lists, company names and sentiment. It extracts the sentiment and the company names and adds that to a dataframe and writes the data frame to a CSV file. Now from what I see here, it actually does not include that parameter. Like override equals true. But yeah, okay, now I've added this here. Like, do you see anything that could be improved from this output? I'm going to give a quick hint here already. It is basically going to perform everything here using that Python tool. So for example, when we are creating our sentiments, it is going to create very simple rules to do that sentiment analysis. Same goes for our company names. It's going to create a very simple rule for those company names where actually we're not making use of the large language model anymore to do this. So that's kind of a downside. But there is a solution for this and we're coming back to this later in the Notebook. But the cool thing is we have this data frame. We've written it to a CSV. So if I go and look at my file system here, I now see that this CSV popped up in our file system. So the next step we want to do now that we have this CSV, is to basically open up the data frame and check what the results look like. So this is going to be very straightforward. We're just going to open the data frame. So from Pandas, we import Pandas under its usual alias, which is pd. So we do import Pandas as pd, then our data frame is basically div. So we do Pandas readcsv, and we want to read financial analysis CSV into a data frame, and we're just going to print that data frame. So one of the things that you might notice here is, first of all, this company name is weird. And these are as well. The reason for that is, as I said before, that Python tool generated very simple rules to perform like this company name extraction and to perform that sentiment analysis. So we actually don't want to use the Python tool for that one, but we do want to use a Python tool for stuff that we want to do with code, such as creating a data frame, writing it to a CSV file, and many other cumbersome tasks that we can add to this whole process. Now, from the sentiment analysis point of view as well, I think this one should not be negative. So we see here these quarterly revenues actually increased and it is defined as negative. Now, the reason for that as well is because if we look up here, it just, It said that if. Yeah, basically just very simple rules. That's. If loss is somewhere within that headline, it should be considered as negative. No, I'm not even sure. Oh yeah. So up from a loss in 2008. So this should be positive, but it's considered negative because of these simple rule sets. So we want to look at how to improve this now before we're going to improve on what do we want to have performed with Python and which tasks do we want to have performed by the large language model, we're going to touch upon one more concept first, and that is another way of improving our. Performance of these prompts, and that is by doing few shot learning. So the next step is to add few shot Learning to our prompts, Few shot learning is basically just adding a couple of examples to our prompts. So for few shot learning, yeah, we have few shot learning. And that goes together with stuff like single shot learning, zero shot learning, where zero shot learning is basically we don't add any examples. Single shot is we add a single example. Few shot is we add a couple of examples. Now those examples could be something like explicitly saying this, this, and this is considered positive. In my opinion, this, this and this is considered negative. It could also be just pasting in a couple of headlines with a label in there. So specific observations with a grand truth that we add to our prompts that will actually increase the model because it will have an increased understanding of what we mean. With our prompts. So for example, when we're talking about sentiment analysis, there could be that ambiguity of do I mean like emotional sentiment analysis or financial sentiment analysis. Like I could say something like I hated that company X's profits skyrocketed. Now from a emotional perspective, that is a negative analysis. I say I hate this from a financial analysis that is considered a positive sentiment, like, yeah, their profits, their profits skyrocketed. So that's a very good thing from an investment point of view. Now that kind of ambiguity, we can solve that with few shot learning. So we're explicitly saying, we're explicitly giving a couple of examples to show the model exactly what we mean. But let's quickly create two templates, one without few shot learning, one with few shot learning, and then we can compare the results to each other. So first we're going to create a new prompt template. Without output parsing, this is nothing much new. This is what we have done before. But this time instead of for extracting the company names, we're going to extract that sentiment. So I'm going to create a new prompt template called Sentiment Template. This is a prompt template which again takes these three variables because I want to add that output parser again to have a instant list of sentiments that correspond to all of our different headlines. So first of all we have template and our template itself is listed here. So that is one that we've seen before. Next we have our input variables. Our input variables. We have this headlines input variable. So we have a list with just headlines in here. And then lastly we have our partial variables, which again takes in the dictionary of format instructions, which equals to the format instructions that we had previously. Now we're going to format that template. So we're basically just going to input something to our in that input variable. So we do formatted sentiment template. Sentiment template equals sentiment template dot format. And for those headlines parameters, just as we've done before, we're just going to pass in all of our headlines as well. Then we're going to run that model much in the same way as we've done before. So we're just going to do output model and we're going to fill in our formatted sentiment template and then we can output that, we can parse that output just as we did before as well. And that output will be our final Python list. So that will be our sentiments list as we've seen here. And this is just a same stuff as we've done before. So let's do outputparser parse on the output of our model and let's take a look at the sentiments. So now we see a slightly different approach. We see positive here instead of that negative from the rule based approach. Now it is, it might already be like fully correct. So I'm not sure if we will see an increase in quality when we do this or when we add few shot learning in a second. But just so you know, just in general, this is a better approach and this will yield better results on larger data sets. Because we only have six instances now, our sample is not that big. Now to visualize, to make visualization a little bit easier, we're just going to create a quick function that combines this label with their headlines. And here I also wanted to teach, maybe if you're a Python beginner, teach some extra stuff here, such as a assert statement. So what we want to do is we want to create a quick function which basically takes in the headlines list of strings and the sentiment list of strings as input. And we're basically going to display them in any way possible. Like you can choose how you do it. It could be as simple as just having an f string that deals with both. But first, what we want to do is we want to check if these lists have the same length. And this is for example, where an assert statement can come in. An assert statement will check for a certain condition, and if that condition is not true, then it will throw an error. So it can make sure that if our code runs, it runs as we expect it to. So for example, we're going to assert that our headlines list and our sentiments list are of the same length. Because let's say if something unexpected might have happened in or with the, with the model output, somewhere along the way we want to know. So let's create a quick function called visualize Sentiments and this will take in two inputs. So we need some headlines and we need some sentiments. First of all, I'm going to assert that the headlines list and the sentiments list are of equal length. So I am just going to check if the length of headlines is equal to the length of sentiments. Next I am going to visualize them in any way. Now I would just because of the simplicity, I would just advise to use like a Python string, like a Python F string, for example, and I format it. I prefer to use is. Let's say we put in our sentiments here. So here we're going to have our specific sentiment and we're just going to have our headline here after. But we want to do this for every sentiment and for every headline. So we're going to loop through our list. But I want to have the index. So I could do for I in range of the headline. What is also a possibility is that I do for I an underscore because I don't use this one in enumerate and then I just choose one of the lists at random because I know that they're the same length. And this enumerate will basically make sure that we get two variables here. This will be the index. This will be the actual value of that list. Now I am not using the actual value. So I am using an underscore here to show that I am not using this variable. So what I want to do is I want to say something like positive colon and then the headlines. Now maybe to make it a little bit more clear, let's put that positive or negative or something like this in uppercase in all caps. So we can do with upper. And then we're just going to call that function. So let's do visualize sentiments and we pass along our headlines list and our sentiments list. Let's see. So this is a bit easier to evaluate our output. So we see positive and this, okay, the profits rose, that looks correct negative for this one and we see the losses widened. So that looks also correct, etc. Okay, now we want to add few shot learning to our prompts. So as I kind of explained before, like emotional sentiment and financial sentiment might not always mean the same thing. So that is one example of why we should use few shot learning. But just in general, for most tasks, adding few shot learning increases the actual performance. The only downside to it is that it requires a bit more work from our end on basically building that prompt. So for our example for few shot learning, I just added a couple of examples like what do I Consider positive, for example when our profits or revenue increased, when we're exceeding expectations, when a contract is awarded, etc. When losses are mounting up, blah blah blah, that's considered negative. And if nothing is positive or negative then and yeah, we have a neutral sentiment. So this is our few shot learning and we just want to have this added to our prompt. So let's include this in our code as these sentiment examples. So we start by storing the few shot examples in a variable. So we do sentiment examples equals our examples here. Then we're going to instantiate a new prompt template. We're going to call it. Let's see, we're just going to override the old one. So we're going to do sentiment template is prompt template with the same three with the same three input variables. So we have template. For the template. We want to include the few shell examples as an input. So let's see here. We basically start by getting our same the same normal template here. And let's see what would be the best location to add our examples here. So first I ask it to get the financial sentiment of the following headlines. That makes sense to start off with that. It makes sense that we add it somewhere like here. So first we tell it get the financial sentiment, then we tell the model what do we consider to be correct financial sentiment? And then we're going to talk about the output. So I'm just going to add a input variable here called few shot examples. Next up we have our input variables. So previously we just had the headlines here. Right now we have two input variables, so we have both headlines and we also have these few shot examples. Now for a real life case, you typically wouldn't add few shot examples as an input variable most of the time because you want them to be in there for every prompt. As long as you're doing the same use case, it's only when you're using the same template for different use cases, which is more rare, less occasional. Then you might want to add different few shot examples based on what you're going to use the template actually for. Then lastly, just like we did before, we're going to add like partial example partial variables which takes in the format instructions and we're going to use the same formatting instructions as before. This will also be pretty much the same. So our formatted sentiment template equals our sentiment template format. And now we need to fill in two inputs here. So we need both our headlines which equals our headlines list. And our input is called our input variable is called few shot examples. But the input here is going to be the examples for the sentiment analysis. So here we pass along sentiment examples and now it's just like the same few steps as before. So we're going to run the model. So again, using this output variable, I'm going to run the model on that formatted sentiment template and parsing that output again and overriding basically our old sentiments list. So I want to do output, parser, parse output. Okay, good. To have a very quick overview of what the output now looks like. We have just created this function to quickly visualize, so let's start using it again. So we do visualize sentiments. I believe headlines was the first one. Yeah. And sentiments was the second one, sentiments. So let's see. Now in this case we do get the same results because the results were already looking pretty good. But in general, just expect on larger use cases on larger data sets expect to be for more answers to be correct once we start using this fusion learning, at least if we're doing it the right way. All right, now that we have covered fuchsia learning, we can address the problem that we had before. The problem that we had before is the fact that our Python REPL tool was basically executing tasks that we didn't want it to do. So we did not want to extract company names or to analyze sentiment using simple rule sets in Python. That's what we're using large language models for. But the simpler tasks, which are still, I would say, cumbersome as a, as a developer, like just setting up another data frame, writing locally. This is a simple example, but we could replace this with much more complex tasks. That is what we want to use code for. So how do we deal with this? Basically an easy answer is multiple large language model calls. So we make multiple requests and we've already done a few of them because we've done calls to large language model without the usage of any tools. To get that company names list and to get that sentiments list, now let's just ask it to use those lists and perform the task that actually requires the Python tool. So for this we change our prompt a little bit. Now I am, I see a small mistake here. Create a data frame with two columns. Should be three columns. Let's see how it reacts to this. So again, company name, sentiment and headline. We fill that data frame with the following lists. So we have the lists of all the data. We're just asking it to generate the pandas code. Very simple for now. And just writing it to a local CSV. This time we use with Parsing as a file name. So code wise, again, super simple. We're just running the agent executor. This is more about how to write prompts than writing code, to be honest. But just let's run this. Now I am curious what's going to happen with this. So I think it will generate a data frame with three columns. Yeah. So it is smart enough to know that I pass along three column names. So I actually want to have three columns. So yeah, look at that. I have a data frame with three columns and it is being written locally. So if I look at my file system here, I see that I have a new file here. Financial analysis with parsing. Now again, let's take a look at this data frame. So we're just going to do a very quick look with Pandas read CSV and I want to read in this file and let's just take a look here. So this is looking a lot better than what we did in task five where the sentiment was extracted using rule based approaches. And the same goes for the company name. Now we get like correct company names, we get the correct sentiments, we get all the headlines with it. So this is exactly what we wanted. Good. Now this basically rounds it up, what for our financial analysis. As a last task, let's look at using the Moderation API for OpenAI. Now for that moderation API, I have been on a subreddit called WallStreetBets to kind of stay within that financial topic. But I had to find some examples that could trigger the moderation API in some ways. Now there is a content warning here. The comments are stored in Reddit comments txt and printing these comments is completely optional by default and especially like within this recording. We're not going to actually do that, but if you really want to, you could. It's just like as we say here, these comments are chosen to be offensive. So if you do not wish to actually read these comments, do not print them out. But we're just going to take a look at the output. Good. So. So first of all we're going to read in the text file the Reddit Commons text file, much in the same way as we did with the financial headlines. So again, we're doing with Open Reddit comments txt as read format and we do this as data comments equals data relines. Now, optionally we can print these comments, but we're not going to do this at this point. We're going to pick a random comment. So let's just do comment and we'll start with the first one, so we can mess around with them a little bit. We can try with index 1, index 2, 2, 4. We have five comments in total, so you can try any number between 0 and 4. And just check if you get like, let's say if you get flaggings for different categories. But basically what we want to do is we want to send this comment, which is just a string. You want to send it to the moderation API and see what the output gives us. Now, from a generative AI perspective or from a more operational perspective, how we're going to do this is basically when we're sending prompts to the model, that prompt would be sent to this moderation API. So that prompt is in the end, just like a long string. We're going to send that prompt to the moderation API, check if everything is okay, and then we're going to send it to the actual model. So let's see, we want to output. We're going to assign the output to moderation output. So that's what we're going to start with. Moderation output is going to be. We're just going to use OpenAI moderation create. We already imported the OpenAI package at the start of our notebook, so this should be able to work straight away like this. And then our input is simply going to be our comment, and we just want to print out that moderation output. So what we see here is the output, the comment itself is being flagged. More specifically, it's being flagged for harassment. Now here we see the different category scores. So if you want to have like different confidence levels, etcetera, it's also in there. And finally we have a single value. Flagged equals true. And that is just to make it a bit easier, if any of these categories turn to, through this flagged key value pair, the value here will be true as well. So in an application, we're just going to check like, if flagged is true, then we might check like, okay, what category is actually it's being flagged for. Or if we just want to keep it simple, we're only going to look at this flagged. Like anything that's flagged, we're just going to block it. We're going to filter it out of our stream and not send it over to the model. Right. So with that, I have to congratulate you on completing this module. I hope you learned a lot on like the basic concepts of LangChain and also like prompt engineering techniques. We've seen some prompt templates, we've seen some prompt engineering tricks like the few shot learning we very briefly touched upon LLM chains. But I didn't want to go too deep into this because the topic is not very complicated. Output parsing and agents and tools are some very, very powerful options in LangChain. Could be very valuable to go deeper into this and to basically look up more and maybe build some projects on this. And lastly, leveraging the moderation API to access the filter. Now using that moderation API is super simple and gladly so you just need to know like when do you need to use it? So basically for AI applications, you mostly want to put that as a filter for anything coming in. Now with that being said, I wish you the best of luck in the following modules and good luck in all the other courses.