Building Chatbots with OpenAI and Pinecone | Become an AI Developer

Hi, I'm James, I'm the founder of Aurelio AI and a software developer advocate at Pinecone Devector Database. In this project we aim to explore the world of chatbots that are able to get information from the external world using rag. Now you've already learned about Pinecone and rag and what we're going to do in this session is really just pair that with a essentially conversational memory which allows us to talk to the chat bot, reference previous messages and have more of a fluent conversation rather than just a single query that returns information for us in, you know, at any one point. Okay, so let's jump straight into it. We're going to be using this data set that I sourced from the Alarm 2 archive paper. Alarm 2, I'm sure many of you are aware, is one of the more recent open source LLMs from Meta. And because it's very recent, that means most LLMs are not aware of any information about llama 2. So it's a very good example of where we might want to use rag. It's where our LLMs knowledge is just not up to date because you, you can only train it every so often. And when you train it, it's one expensive, requires a lot of data and you know, you need to pull in all that data and it's, it can be pretty hard to do and especially very hard to keep that information up to date. So that's a very good example where moment you use rag, keep your LLM or chatbot up to date with the latest information. So we're essentially going to be building this here, but as well as what you can see here, we're also going to have conversational memory. So we have a memory of, you know, what has been happening within the conversation. So we should be able to see something like this. Now we're going to ask what makes Armature special. We take that to our embedding model, we query our vector database and we retrieve relevant context. They help us alongside the query to produce a retrieval augmented query. We send that to our LLM and then we should be able to get an answer that is aware of what llama2 is and can actually point to specific information about it as well, which it has Learned from the Llama 2 paper. So let's jump into how we're going to do do that. To begin, you are going to need a OpenAI API key and a Pinecone API key. I if you've been following along the previous projects in this course, you, you will have already Done that. Now, before we jump into the code, let me just talk a little bit about what libraries we're going to be using. So you can see here we have Lang chain OpenAI hugging face data sets, Pinecone, and this tick token library. Now, LangChain is a incredibly popular library. It essentially acts as almost like a wrapper that allows you to pull in multiple components that are useful for building LLM applications. Like a chatbot. We have OpenAI. Obviously, we're going to be using the OpenAI for both our embeddings and also the generation part of our pipeline. Hugging face data sets is where we're going to be getting that llama2 archive data set from. We have Pinecone client for our vector database, and tick token is also required when we're chunking our text and for other things where the token count. So the length of our text according to the LM or the embedding model is needed to be calculated. And we use tick token here because that is the tokenizer that the OpenAI GPT models use. So we can jump into setting up our chatbot. So we're going to need to Initialize this chat OpenAI object from LangChain. So we're going to be using a lot of LangChain abstractions here. So from LangChain we want the chat models and we want to import the chat OpenAI class. We can then initialize this chat OpenAI and all we need to do is pass in our model name. The model name, you can either put model or you can put model name is GPT is GPT 3.5 Turbo. Okay, so this is at the moment the standard chat model for, for chatbots, you can also use GPT4. It's just slower. And obviously when you're having a conversation, that's something that is very important. You want to have a fast, responsive chatbot. So for the most part, I always kind of recommend 3.5, at least for now. So that will initialize our model via linechain. One thing that you'll notice here is that we haven't passed in our OpenAI API key, and that's because we already have that defined within the OpenAI apikey environment variable. I have set that in mine over here. So I have my environment variables edit, and you can see that I have OpenAI API key in there. Cool. So let's have a quick look at how these chats are going to be structured. So both 3.5 and GPT4 are chat models, which means that they structure the, the inputs like this, right? So you have a system message. This system message is how you essentially sell the behavior and instructions for your model. So any, like, overall instructions go into here. You have your user measures, which are obviously from the user, and then you have your assistant messages, which are responses from the chatbot. Now this is, you know, how it would look in a chat log, but when we're actually passing these to the API, it's going to look more like this. Okay? So this is if you're interacting directly with the OpenAI API. So we have the role which represents obviously these here we have system, user, assistant, user, okay? Then we have the content, which is obviously the text content. For each one of those messages in line chain, there's a slightly different format. Okay, so they have a further abstraction. And the idea is, with these abstractions that linechain has, the idea is that you can switch between different chat models. So not just OpenAI, but other chat models as well, and use the same structure. Right. In the end, you know, behind everything, they're just going to reformat things for you depending on which model you're using. So here is how we create this and this, but within LangChain, so we have this system message, human message, AI message, right? These are the abstractions, obviously human Maps to user, AI MapsU assistant. So to create a conversation, we are going to need those three message types. So first what we need to do is just import those, And we can import those from LangChain schema, and then we just import each one. Okay, so we have system message, human message, and AI message. Now we want to do is essentially create this, our code. Okay? So we want these messages. So we can actually, you can see the messages are the same. So you can actually just take this and put it into here. Okay, cool. So let's ruin that. And now what we can do is actually chat with the chat model that we earlier set or initialize. So this model here we have chat. So let's take that and pass these messages into it. So we're going to pass it into a response object and we're just going to pass the messages into, in their current format to that endpoint, And let's see what that gets us. Okay, so you can see that we get this. So we have a AI message here. So it's already formatted in this same response or same format that we have here. And we have the content. String theory is a theoretical framework in physics. Aims to explain the fundamental nature of particles and their interactions and so on. Right. So we can see that this sort of format is working. But let's, let's format this a little bit more nicely. So to do that, we're just going to print so that we can see the new lines that we have here more nicely formatted. And we want to print the response content. Okay, See, the content is here. Okay, so we get. I think this is much nicer. So we have a numbered list here. Wasn't very clear from before, and yeah, much easier to read. So I think we get a pretty good. Although, you know, I certainly don't know for sure what any of this means, we seem to get what I think is probably a good explanation. So now what we want to do is continue the conversation. Obviously, you know, this is a conversation we want to, we want to have like a back and forth between us and the AI. So to do that, what we need to do is actually take our response object here, which is the AI message, and, and we want to append it to our messages that we created before. So to do that, we can see the instructions here. We want to append. Yeah, the latest AI response to messages. So messages append and we have our response. And let's take a look at what that will give us. Let's see what response is first. Sorry, let's see what message is first. Okay, so we just have these four. Now let's append. And of course, we now have our next message in there with the rest. Great. So what's next? We want to create a new human message. And you see with this message here, we're not saying. And you can see with this message here that we're not saying. Okay, why do people believe that string theory can produce a unified theory? We're saying why? Why do they believe it can produce a unified theory? Right. So in order for the chatbot to answer this correctly, it is going to need to rely on the previous interactions. So that conversational memory in there is going to be very important. Okay, so let's just delete this or. Okay, let me create a new one here. So I'm going to assign this new human message into our prompt human message. The content is what we have here. So I can copy this. Okay. And I'm also going to append that into our messages. Okay. And we'll see that in there now as well. So we have our new human message within the chat log. Now let's do the same as we did before. So we will go to here, And then let's also print out the content from that response. Okay, cool. So you'll notice as well, as we keep adding more and more messages into our, like, into our chat here, the response times actually get longer and longer. And that is because you can think of it as with every single token or maybe every word, if you want to think about it like that. The model, the AI model, the GPT 3.5 model, is looking at every single word like one after the other. So the more tokens we have within the input, because we're passing in all the previous messages with each new request, the more tokens that we have in there, the longer it's going to take GPT 3.5 to run through all of those and then start generating an answer. This also applies if you are producing a longer answer, it's going to take longer. If you are prompting GPT 3.5 to just generate very short messages, it's going to be faster. So something just to consider now. Let's have a look at our response here. So physicists believe that string theory has the potential to provide a unified theory for several reasons. So great, it knows we're talking about string theory and it's answering our question about the unified theory. Okay. And those of you that are interested, you can read through that as well. So that all looks good. Let's now move on to talking a little bit about hallucinations. Okay, so we've, you know, we've covered this a little bit already in the previous two chapters, but with, we have our chatbot, it's not connected to the outside world whatsoever at the moment. So that means that it is just relying on the knowledge that is learned during training. Okay, so we have this sort of scenario here, right? This, you know, there's the outside world. It has no idea about the outside world. All it understands is what it has seen within the training data set. So it means that GPT 3.5 and other LLMs are going to perform badly on quite a few different types of questions. So more general knowledge or the, you know, like the questions we're asking about string theory, unified theory and whatever else, it knows that, okay, that information hasn't really changed or maybe there are updates within that space, but it can give us answers to, you know, those questions. But in other scenarios, it's going to perform much worse. Right? So when chatbot doesn't know about recent events, so, so for example, with recent events, it will have absolutely no idea, right? You ask, you can even ask GPT 3.5 about COVID It doesn't know about COVID because the training data cutoff for at least Those models, I believe, was September 2019. All right, so it's a few years ago already. And you know, they, they just don't know anything that has happened since then, which is, you know, it's quite a lot. So if you want a, a chat bot to respond accurately to something like the weather in your city today, it's, it has no idea. Right? It can't answer that. It also can't answer questions about recent code, recent products. Right. So if I ask it to tell me about the LM chain in line chain. Right. So this is a particular object within the line chain library. It will have no idea. It doesn't even know what line chain is. Okay. If I would also want to ask you, maybe, you know, what was the latest course that was released on Data Camp? Again, it has no idea. It might know about some courses from, you know, 2019, but you know, times move on quite quickly, so that's probably not going to be that useful for us. And then another like, and then another use case that I see all the time is questions about confidential corporate information that isn't on the Internet. Right. So think about your internal documentation within an organization. It's all internal. GPT 3.5 and 4 have absolutely no idea about it. Neither do any other LLMs hopefully, at least. So yeah, that's another scenario where you LLMs just aren't going to perform. They can't, they don't have that knowledge. So let's try, let's try and ask about the Llama two large language models. So I'm going to append the latest response that we had up here. I'm going to append that to our messages. It's not so important for this use case, but it'll be weird if we like ask a message, if we ask a question and then there's no answer and then we start asking about other things. So let's just make sure we have that in there. You can see how long our messages are right now. Okay, so we now have seven messages in our conversational history. And now let's create a new one. So I want to say, want to say this up here. What is so special about llama 2? Okay, so our prompt is a human message, remember? And here we go. We're going to append that to our messages as we did before. We're then going to send that to our chat model and we're going to print a response. Let's see what we get. Okay, so it says, I'm sorry, I couldn't find any specific information about something called llama 2. It is possible that you might be referring to a specific context or topic that I am not aware of. Okay, okay, so. Okay, so you will actually see this happen quite often with like. I can show you this example here. All right, so this is when the GPT4, right? So the most powerful LM in the world. You know, whilst I'm speaking right now, I asked it, how do I use the LM chain LangChain? Super basic question, right? And it gives you like just hallucinated. This is back when GPT4 was first released. Langchains are popular. No, sorry, LangChain is a blockchain based platform that combines AI and language processing. LM chains, a token system used. LM chain is token system using LangChain. Basically all of this stuff is completely wrong. It's just making stuff up. Now. Since then, OpenAI have modified the response that you'll get from GPT4 and M 3.5. It will now, it will literally now say basically this, right? And this is you. You see this happen quite a lot. So when for example a new model is released, like Llama 2, when you, when that's first released and you ask GPT 3.5 or 4 about Llama 2, it would actually give you like a hallucination. You're going to start talking about llamas. And then very quickly, I'm sure, sort of behind the scenes the OpenAI team or engineers are like patching these things when they see them pop up. Right? So. Here we're getting this response. It's essentially like a guardrail because if llama 2 is mentioned, it's a guardrail that say it tells GPT 3.5 or 4, you don't know about this, don't respond. And they did the same with the LM chain, Lang chain and so on. And you see this happen all the time. So they're very quickly like patching these things so that it just doesn't respond with like popular things are happening that it doesn't have like training data on yet. So let's try that again. Let's. We can take this. And actually I will just append a previous response to this. So, yep, append the latest AI response and messages. Then we want to say, can you tell me about the LLM chain and LangChain? We'll see what it says now which is going to be slightly different. We append that and yeah, we go through. So let's see what happens. Okay, so you can see that it's kind of producing the same response. Right. So you can see here, I'm sorry, but I couldn't find any specific information about something called this. I couldn't find any information about specific concept called this. So this is that sort of guardrail type behavior in action. It. Okay, so we're going to take this description here. Right? So this is, it's a copy and paste from I think the line chain docs on one LM chain is. Okay, so it just explains everything. Now in this we have 1, 2 and 3. 3 records. Right? So LLM information, it's 3 records. So you can see if we do LLM chain information, we'll see the length is 3. Cool. What I want to do is concatenate these all into a single string and we're going to assign it to the source knowledge variable. So source knowledge. And what we'll do is just separate each one of these with a new line character. So join LM LM chain information like so. And let's see what we have. Okay, so we now have this. Now what I want to do is actually pass this into our query alongside our question. Okay, so we're going to do, we're going to augment our prompt here. So we're going to create that augmented prompt. So let's take this. Into here. And what I'm going to do is define a query. The query is going to be same as what we saw before. Can you tell me about the LM chain in line chain? So come down to here. And we are basically we're going to pass in our context. We're going to pass in our query and we're going to see how, how this might perform. Okay, let me just show you what this looks like quickly. So we now have our augmented prompt. Let's print that. Okay. Using context below unto the query context and then our query. Okay, so this is a very standard structure that you'll see in when you're prompting LLMs and adding in extra information. So we have our instructions at the top, we have our context or external information in the middle. Then we have our query at the end. Now what I want to do is add this to our messages. So what I'm going to do is actually take a look at what we have in there right now. Okay, what is the last one? So can you tell me about the lmchain in LangChain? I'm asking that question again. So what I want to do is actually just replace that final message with our new message. Okay, so it's going to be human message and Then the content is going to be equal to the augmented prompt. Okay. So yeah, let's. Let me print that out so we can see that it's changing print messages minus one. So the final message. Okay, We'll just replace it with this. Now let's try and see what will happen. So I'm going to take the code that we used before to prompt our prompt, our chatbot. Here we. Here we go. This, See what we get. Okay. Now. Okay. LLM chain is a type chain within lunching framework for developing applications powered by law. Sorry, by language models. The LM chain, short for language learning model chain. So this is actually. Here is a good example of hallucination. It's. It's large language model chain. Okay. LM chain, it's kind of made that up. I'm not sure. Y. And we can see it includes the following elements. Prompt, template, model, optional output, parameter. These things are actually true. Okay. So yeah, we can see that this is actually feeding us useful information. Now we can also modify our augmented prompt if we'd like to avoid this type of hallucination that we're seeing here. So if we. I can show you how to do that quickly. If we take a look at our augmented prompt, we say use it in the context below. Answer the query. We'll just tell it. Right? Do not make anything up. That's basically what we're going to say. So if the answer or if some information is not provided within the context below, do not include it. And if the answer, and if the query cannot be answered with the below information, say I don't know. Okay, so just prompting this to not hallucinate. So let's try that. Rerun all this. We have our. So this is the old message that we had. I'm going to replace that again with the new one. Okay, so now slightly different. And now let's just try again. Okay, so we get better here it says language model chain, which is like this part of it. Okay, that's fine. Not necessarily wrong. It consists of a prompt template, so on and so on. Right. So we can see that we're getting more correct information here. But again, like, you know, there's definitely things we could improve and that's sort of where your prompt engineering would come into play. Nonetheless, this response compared to what we were getting before is far, far better. And the reason for that is because we augmented our prompt with that additional information, the external information. But we obviously, like, we can't do that with every single prompt because we, you know, how do we know what the user is asking about, you know, we. Okay, we know what they're asking about. But I'm not going to sit at a computer and pull in information for every user's prompt. And I don't think anyone else is either. So we need a way of kind of automating this augmentation process. And obviously that's where RAG comes into play. So what we're going to do is just import some data. We're going to be using this data set here. So we're going to do from data sets, import load data set. Data equals load data set. We're taking the training split and we can just see what we have in there. Okay. So it's going to take a moment to download. Okay, cool. So we have all of this stuff here, we have almost 5,000 rows. Let's take a look at what one of those might look like. Okay, so we can see we have the trunk here, we have a summary of the paper, so on. Right. So if we go this chunk is like a very small, you know, you can see it's like paragraph to two paragraphs worth of text from the paper. If we go to the next chunk, right, we'll see that the title of the paper will stay the same, the summary will stay the same and so on. Because it's coming from the same paper. Let's see. Okay, so did that change? I didn't even see. Okay, yeah, so you can see that the chunk changes, but the this does not because they're the same paper. The trunk ID will also change as well. Right. One thing to also note is that we have our overlap in there. So we have this January 2011 abstract we print, present a fast, fully parameterizable GPU implementation, so on and so on. Let's see if that is near the end of our previous chunk. Okay. And it is January 2011 abstract, fast, fully parameterizable GPU implementation, so on and so on. So we have that overlap in there as well. Now, as a quick summary of what we're looking at here, it's Source from the llama2arxiv papers. It's a collection of academic papers from arxiv, which is basically where they put all their research papers. Mainly we use it for tech research and particularly when there's a new LLM or other AI research, we'll usually find out about it first from there. Now the way that this was sourced means that we will actually have other papers in there. So we can see that this paper is called High Performance Neural Networks Visual Object Classification. What we've done here is we've basically taken the Llama 2 paper, we've looked at what other papers were cited within that paper, and we've also pulled those in as well, just so we have like a more rounded data set of kind of different research. But all of it is so pretty directly relevant to the Llama 2 paper. So we need to build our knowledge base. We've seen how to do this before. So we'd initialize our connection to Pinecone, create an index, initialize OpenAI's embedding model with LangChain and populate the index. Okay, so let's go ahead and do that. So we need to import OS because we have our Pinecone environment variables in there and we also need to import Pinecone, we initialize our Pinecone client or authenticate with it. So we need our API key. That's going to be OS Environ Pinecone API key. And we also want our environment variable. So environment OS environ it would be Pinecone environment. Okay, so I just want to check that these actually do line up to the environments I have set in here so we can see and check. So, yeah, I have Pinecone API key and Pinecone environment. Perfect. Run this. What happened? I may have just restarted the kernel, so I'm going to go and just rerun everything again. It. Okay. And once that is done, we will create a new vector index in Pinecone. So going to import time so that we can just wait for our index to be ready once we've initialized it. I'm going to give my index name. You can call this whatever you like, but just try and make it descriptive as to what you're using it for, so you can kind of come back and recognize what it's actually for. And I'm going to do if index name is not in the Pinecone list indexes, so we saw this before in the previous chapter. Then I'm going to create my index. So then I'll do Pinecone create index. Now remember, we have the index name parameter in here, But we also have we need the dimension parameter and we also need the metric parameter. So dimension is going to be equal to the dimensionality of R002, which is 1, 5, 3 6. And the metric must align to the metric that other 002 uses again. Now that can either be dot product or cosine. I'll go cosine here. Then what we want to do is wait until the Pinecone index is Ready. So I'm going to do while not pinecone describe index, specify which index status is ready. While that is not the case, we are going to wait the time sleep one. Once that is ready, we're going to do index equals pinecone index index name again. Okay, so we're going to initialize our connection to the index. So we'll wait a moment for that. I'm going to create a new cell it. Okay, Now what we can do is we'll do index, describe index sets and we should see that this is currently empty. Okay, so yeah, the total vector count 0 because we haven't put anything in there yet. All right, cool. So we have our index. That's ready. Now what we need to do is initialize our embedding model. Like I said, we're going to be using the R002 model from OpenAI and we're going to use that via the line chain library. To do that, we need to import the OpenAI embeddings class. We can do that like so. So we do from LangChain embeddings. We want open AI and we want to import OpenAI embeddings. Then we initialize our embedding model, which is going to be OpenAI embeddings. And we just want to pass in the model name here. So model text embedding order 002. Using this OpenAI embeddings abstraction, we can create our embeddings like so. So we're going to have some, some text. We can write anything you want here. So this is a sentence, and this is another sentence. Okay, we get embedding model.embed documents and then we just pass those into there. We can do text equals text or just pass the text in directly. We'll get our response from that actually. And let's just print out what we have there. So we have our response and we also, I want to show you the length of a single one of these vectors. So we do response zero. Okay, let's see what we get. And you see that we have two vectors because we have two chunks of text here. And each one of those is of the dimensionality 15, 36, which is the output dimensionality of order 002, which, you know, we covered before. All right, so that looks great. Now let's see how we want to use all of this. And we want to populate our pinecone index in this bit here. So first thing I'm going to do is I don't like to wait for something and not See any progress happening. So I'm going to import tqdm so we have like a progress bar which shows how far through the indexing process we are. Then what we want to do is set a batch size here. So this is how many records or how many chunks of text we're going to send to our embedding model and then also over to Pinecone in one go. We want it to be higher so that we're making the most of parallel computations. But at the same time we don't want it to be too high that we can't fit everything within a single API call. We should also convert our data set into a PANDAS data frame. So we're going to do Data Equals Data to Pandas, so it converts everything into a PANDAS data frame. You can also set this in a different variable if you don't want to overwrite things. But we're not going to be using the. And this version, like the Hunger Face datasets version of it again. So it's okay here. When you say for I in tqdm, then we set the range. So we're going to go from zero up to the length of our data set. And we're going to do that in batches of batch size, which is 100. Then we want to find the end of our batch. So it's going to be what is the lowest number between I plus the batch size or the length of our data set. Okay, we'll get the records for our for that batch. So Ilock here from I to I and. Let's create our IDs first. So we're going to need IDs, the text which we'll convert into embeddings, and also the metadata. So IDs we can create them by taking. We have actually, we can see here, Just take a look at that first record. Okay, we have the DOI and we have chunk id. We're going to put both those together because the DOI is the paper identifier and that doesn't change. Look at this. It doesn't between. Change between papers. Okay. So actually easier way of seeing this would be if I do this. Doi. Diy. Okay, so we can see between those, it doesn't change. So we can't just rely on that id. We need to merge with another ID and the other ID that we do have here. So if we take those, create this, we can take a look at the trunk id. Okay, so then we have, we have something that changes between chunks. So we're actually just going to merge those Both together. So we'll do XD DOI followed by X chunk id. There we go for X in. Batch. Dot IT rows. Okay. So iterating through those and we'll also get a index value from that. So I'm just going to replace that with j here. Or we can just have this parameter here. Basically means. Right. We don't care about whatever is in there. Okay, now we can copy this and we're going to do something similar for our text. So take our text here. And yeah, the only difference is that rather than this, we're going to go with X chunk. Right. So that's the actual text that we have. Then with that we can create our embedding. So we're going to do embedded embed, model, embed documents and pass in our text. And then what we want to do is the same for the metadata that we'd like to keep. We don't want to keep everything, so we just want a few items in there. We can take again another look at what we might want. So our data, we have the chunk. So the actual text, we do want to keep that because we're going to be returning, returning that via our query in Pinecone. We might also want the source to see where it's coming from and maybe the title of the paper as well. You might also want to include other things depending on your use case. So we'll do the text that is going to be. We saw it already. X chunk. We also want to have the title, Title. And what is the other thing we'd like? We would like the source in case we'd like to link to it later on. Okay, cool. So we have those and obviously we want to get each of those from our iterators, so batch iterators. All right, cool, that looks good. Once we have all that, we are going to zip everything into like a. A generator function that generates tuples containing an id, the embedding and the metadata for each record. So it will look kind of like this. So you'd have your be a list or an iterator. We have ID1 embedding 1, metadata 1 and then you'd have another one. So another tuple that would come through here. This would obviously be number two. And so on and so on. So I'm going to do index, upset our vectors, We zip our IDs, the embeddings and the metad. And we just run that. So that will take a moment because we're adding all this data you can see here. It shouldn't be too Long. So we will skip ahead to when that is complete. Okay, so that is almost done. We're gonna, once it is done, take a look at our index and we're going to see if anything has changed. So we can now see that we have those vectors that we mentioned before in there. Cool. So now let's move on to, you know, how we implement the RAG component within LangChain. So to do anything with Pinecone within LangChain, we need to initialize what's called a vector store object. Vector stores are stored within, so do from LangChain and vector source. And we're going to be using the pine cone vector store, like so we will have a text field within our metadata. Right. So that is text. We set that earlier when we did this. So we need to make sure those two are aligned. And then we initialize our vector store object. So vector store equals pinecon. And in there we need to pass a few things. We need our index, which we initialized earlier. We need the embedding model, but we need the function within that embedding model or method that embeds queries. So we do embed query. Embed Query vs. Embed Documents. It's essentially this will take a single text string or whereas embed documents takes a list of text strings. That's the only difference. And then we have our text field. Okay. And with that we have initialized our vector store. Now let's try and search with this query through our vector. So, so we're going to take this and we're going to do vector store and do similarity search. We have our query, so text, and we want to return the three most relevant records. So run that and we can see that we, we get this. So we have our document here. This is number, number one. Here we have document two, and then we have document three. Okay, cool. So like reading all this, it's kind of hard to kind of pass and, and you know, try and understand especially, you know, these chunks of text here, which, you know, there's no spaces or anything. Fortunately, an LLM can actually handle this sort of messy text and get us an answer much quicker than we could get ourselves. So let's see how we're going to do that. So we're going to connect our LLM up to this RAG pipeline or this retrieval pipeline that we have built. So I'm going to define a function that will just allow us to do this more easily. So this function first one is going to be to augment our prompt. We're going to take in a string and we're going to augment it using what we saw before or using a mix of all the things we've learned so far. So the first thing we want to do is get our data so we get our responses or our results. And then we want to create our source knowledge. So we did this earlier. Remember it was just joining all of our, all of the documents that we've received into a single string. So for this we would do we want the page content? Right, because if you look here, the text we have is within this page content variable. So page content for X in and we want our results. Okay, Then we're going to augment our prompt and we can take what we use earlier. So where was that prompt? Here we have this. So let's take that and we're going to apply it here. Okay, cool. So augmented prompt, we have that and then we just return it. So return augmented prompt. Now let's try with our query again. So augmented prompt, we pass in our query from earlier. Sorry, this is augment prompt, not augmented prompt, which is a variable we defined earlier. And also here. So this needs to be augmented prompt, not augment prompt. Okay, cool. So now we have that, well, augmented prompt in there. You can see it's kind of messy. So we can print if we want to see that a little more clearly. Okay, so we have using the context, so on and so on. We have our context here and then we have our query. Right. So let's see what our chat model can do with that. So as before, we're going to, we're going to create a prompt. It's going to be our human message, remember content. And our content is going to contain this here. So augment prompt query. We're going to append that to our messages or append like so. We're going to send those to our chat model here and we're going to print the response. Okay, let's remove that. So let's see, let's see how it performs with our new augmented query. Okay, so based on the provided context, it's mentioned that Llama 2 is a collection of pre trained, fine tuned large language models and then so on and so on. Cool. So we're getting a good answer. We're also with our sort of hallucination reducing prompt, we're getting it to be more cautious. We're saying. Okay, saying. However, it's important to note that context provider is limited and there may be additional information about Long 2 that is not included. So I think that's pretty good. It's giving us a little bit of a, you know, this is what I know, but I'm. There may be other things that I don't know. So, you know, you should, you should look into it a little bit more. So I think that's, that's pretty good. So let's try some more llama2 questions. What I'm going to do is we're going to create, we're going to create a prompt and stuff, but we're going to do it without the augmented prompt initially and then we'll add in the augmentation after and just we can compare, you know, what the results are like between the two. So I'm going to pull in this question. So I'm going to do prompt equals human message. Again, as we have done already, we're going to have our content now. Content this time is going to be about the safety measures used in the development of Valama 2. Right. What we're going to do is we're going to send this prompt plus our messages over to our chat model, but we're not going to append the prompt to our messages because I want to, you know, I want to change and just see what sort of difference we get. So we're going to do messages plus prompt like this. Okay. So we're just going to append them within the, within the chat function itself and then let's have a look at the response content. Okay. Okay. So based on the provided context, not mentioned, but not explicitly describe. Right, so it mentions that. So one thing to note here is that we have those previous messages. So it now knows what Llama 2 is, but it doesn't, doesn't know, you know, what the safety measures were because they were not included in the previous results. So, okay, let's try again, but this time we're going to include our augmented prompt. So we're going to come to here and we're going to augment this. So augment prompt on the function, not the instance. Let's see what we get. Okay, so just ran that and we can see. Okay. According to provided context, safety measures used in development alarm 2 include safety specific data annotation, fine tuning, red teaming, iterative evaluation, so on and so on. So now we're getting the correct information there. So it's giving us a good answer rather than the answer before where it didn't really know. Okay, so we've finished this chapter. To summarize what we've learned today, we can think about how we have been able to create like an actual chatbot by simply appending messages as we sort of go through the conversation to our messages list and actually have a conversation with an LLM using that. We then again saw how we would index all of our documents, our knowledge base, into Python convector database using OpenAI embeddings. We've also seen how we can one add context to a to a prompt in order to sort of augment that prompt and get an actual good answer from it. And also how we can kind of reduce hallucinations a little bit by just simply telling it to admit that it might not know something. We've then seen how to take that sort of simple augmentation, pair it with pinecone and create an actual retrieval pipeline so that we can augment our queries automatically. Then finally we pulled all that together into a full reg pipeline via the line chain library to create a chat bot that can retrieve information and answer our questions more accurately. So that's it for this chapter. Thank you very much for joining me on this and enjoy the rest of the course.