Retrieval Augmented Generation with PineCone | Start Building AI Tools

Hello there, my name is Vincent. I'm a software engineer at Datacamp and today we'll be talking about using Retrieval augmented generation to search a movie database. So retrieval augmented generation, what does that actually mean? Let's start with a problem first. So you've probably worked with LLMs. If you're doing this code along project, you've probably seen them in previous projects. What you might have noticed is that LLMs or large language models, they are great at auto generating text and completing your sentences. They're a little bit less great at providing factual information. Now they do pretty decently in that context, as long as you ask them for facts that are before 2022, for example. The reason is these LLMs, they can actually only give you information about data that they've trained on. And at a certain point they stopped training. And after that fact, actually after stopping training, there's no way that the LLM can know information that has been facts that happened later on. So there's a few solutions for this problem. For example, you could retrain the model, but as you might know, it's actually very expensive to train these models. Another way to solve the problem is to use a technique called fine tuning, where you don't fully retrain the model, but you kind of fine tune it to new data. If you've tried this, you'll see that fine tuning is actually a technique that you can use to teach an LLM to behave in a certain way. It's actually less good to teach new facts to an LLM. So all of these techniques, they're kind of suboptimal. And that's why actually a lot of systems use something called retrieval augmented generation. And what does it actually mean? It means that instead of retraining of fine tuning, while what you'll do is before you send your prompt to the LLM, what you'll actually do is first send your prompt to some search engine, for example, using embeddings that will give you back some relevant context and you include that context into your prompt itself. Of course, you need to make sure that you select the right context because the prompts have a certain context length. That's a limitation, of course. So in this project, what we'll do is actually we're going to ask create a question answering bot, and we're going to provide that bot with information about specific movies and even movies that were created after September 2021, which is basically the knowledge cut off for ChatGPT. So that's a brief introduction. Let's jump right in. So let's scroll down a little bit. We've talked about his introduction, a few technicalities. For this project, what we'll be using is OpenAI, so we're going to be using GPT 3 and a half Turbo to provide the prompt completions. We're also going to use OpenAI to provide the embeddings, and we'll talk about vector embeddings later on. And in this project, we're actually going to use Pinecone. And Pinecone is a vector database. This is a database where we can store these embeddings and then search through them later on. And to kind of glue it all together as a tool. We're going to use LangChain, because LangChain has a lot of tooling around first, like sending requests to Pinecone to get some information and then send that along with a prompt. It's all pretty conven, as you'll see later on in this project, the data we'll be using, as I said before, we're going to search through a movie database, and the data we'll be using is going to be the AMD Movies and shows with descriptions data set. There's a link here as well. So before we get started, one thing that you will need to set up is, as we said, we'll be using OpenAI and Pinecone. So you'll need an account on both of them and you'll need an open API key for both of them. No worries, though. There's actually in your workspace, if you click on the file step, you should see a Getting Started, this Getting Started notebook there. And this actually contains a detailed guide on how and how to set up everything you need for this workspace. If you go through this, the whole guide, what you should end up with is in your environment panel. In the workspace, you should end up with environment variables, and they should be called something like datacamp AI Projects. And there should be a green button next to it, which means that the environment variables are connected to your workspace. If that's all set up, you're actually ready to start the tasks. And to start the tasks, we'll start with an easy one. There's a few packages we'll need, and These packages are OpenAI, obviously, to interact with OpenAI, the Pinecone client, to interact with Pinecone. Tick token. So this one is an interesting one. Tick token. As you might know from previous projects, OpenAI uses the concept of tokens. So token is roughly equal to three characters. And there is a specific way to encode words and sentences into tokens. And the tick token package is actually a package that you can use to use the same encoding on text. So in this example, we're going to be using the tick token package to encode some strings and to actually give an estimate of how much it will cost to use OpenAI for the embeddings. Then last but not least, we'll be using LangChain, the very popular package that can be used for tooling around large language models and so on. So the first task is actually to install these packages. The reason they're not pre installed in the workspace is because they change a lot. And so we encourage you actually to experiment with newer versions of these packages. However, this workspace has been tested with these specific versions, so they're provided for you. You just need to execute the cell and then you'll find that these packages will be installed. This might take a few seconds. As you can see, it's already done. So that's fine. So we can move on to the first real task. And this is a task here. We will actually start looking at the data. So if you look in the file step, you'll notice that we have this IMD IMDb CSV file and we're going to start it, of course, importing that file. So before we start, I'm going to import Pandas as PD and then we need to create a movies variable. And of course I'll use read CSV typo there. And I'll start with executing this because that will allow me to already have a peek at the data. So we're gonna do a few simple transformations. The first one is pretty simple. You can see that there's a lot of information here. What we will need from this database is, since we're working with LLMs and embeddings, you'll see later that embeddings can actually be used to extract useful information from unstructured text. You can see that there's a description of each movie here. It's a pretty long description, but with embeddings we can actually extract the semantics from those descriptions. We're also going to include some information about the name of the movie. We'll use a primary title column for that. And then we'll also include some information about the genres. And we'll also include information on where to find this movie on the IMDb website. So let's start from the top. We'll rename some columns to make it a bit more clean. And to do that we'll Use the rename method on the DAT Movies data frame. We need the columns field and we need to pass the dictionary. Let's see, we need to rename primary title. I'm a bit lazy so I'm going to just copy that to movie title. Let see. And then description. Copy pasting is a great way to not make too many mistakes, by the way. So let's see what that did. Ideally, yeah, you can see that the primary title is now called Movie Title and description is called Movie Description. Nothing fancy there. The next one. And this is actually immediately something that's very useful about retrieval augmented generation. We're going to be including a source column and what we can do later on, and this is something that will become clear later in this project, is we're going to include a column and that's going to contain a link to these IMDb movies to these movies on the IMDb website. Of course you can see that there's a tconst identifier here. And actually you can build a link for a specific movie. For example, this is the the Silence of the Lamps. So if I go to this one and I copy this identifier here, I have an extra teeter. Oh no, it's fine. This will actually redirect me to the page for the Silence of the Lamps. And what we want to do is we want the question answering bot. We want it to be able to provide the source of the things it's answering about. So what we're gonna do is we're gonna include a source column which is basically just the start of a string. And then it will be appended with the tconst column. Moving on. And I'll do the next two ones in one line. We're gonna filter out some rows. We only want rows that are movie titles. So we'll include title type equals movie. And then we only want the columns we're interested in, namely Movie Title, Movie Description, Source and Jarrus. Let's see. Yeah, I'm actually gonna show the head alone. So here's the five first movies, Silence of Lambs, short Description here and then the Jarvis. You can see the link, it's filled in as link. This is something that workspace does for you. It creates a clickable link from that. And if we click this, it actually redirects us to the IMDP page, but it actually contains these links themselves. So we're going to be using that later on. So we end up with a cleaned up database. So it's time to move to the next task. So in Task two, what we're going to do is we're going to be creating something called documents from this data frame. So what's a document? It's actually a concept of LangChain, and there's not a lot of documentation on this page. But what it does is it's an object that represents some unstructured text, for example, that could be from a book, that could be from a website or whatever, and it puts that in a page content field. And it also could also represent, of course, some rows in this data frame, as we said before, these movie descriptions. It's unstructured text, it could be any length. And we'll put this in the page content of these documents. These documents, they can also provide metadata for each document. You can provide, for example, metadata about movie title or the source. In our case. In our case, we're going to be mainly focused on the source and we'll see later what we'll put in the page content. So what we want to end up with actually for this one is we want to end up with some page content that represents everything, like textual related, anything that's kind of unstructured that we want to have an embedding for. And what we're going to do next is before we create documents, we're going to. We're actually going to be creating a column called page content, and we'll include a string in there that's composed of the title, the genre and the description. So before we're going to create these documents, we're going to do a little bit of data cleaning. We'll see later on why this is interesting to do it like this. But basically it means that when we have, when we include these, upload these, create embeddings for these pieces of text, we can actually search through them. And it will also match, for example, on the title of the movie. It won't only match on the description of the movie. So that's pretty interesting right there. So first we're gonna do some imports from document loaders. We're gonna import the data frame loader, and now we're going to create the new content. So the description says it should contain information about movie title, genre description, and it should look like this. Well, we can do that. We can create a column called page content, print it afterwards. So I have an example. And what we'll do is we'll take. Oh, sorry about that. We will start with a title colon space, and then we will add movies. Movie title is what we call it, right? Yes. And then we Will start a new line as we said before. All right, Then next. We will use the genre movies and then jars. All right, another new line, and then we will add the description Movies. This. It's called Movie Description. All right, and that should be good to go then. Let me try this out. All right, there's an error. It's not Endchain, it's LangChain. Right. That's looking pretty good. We have a page content column, and it looks exactly like what was asked. And this actually contains all the information we need now from the title charts and description columns. So what we'll do is we'll keep only the page content and source columns. So now we don't need this one anymore. And I'll move on. So next we need to load the documents from the data frame into LangChain docs, and we're gonna call this Docs as is required. And we're gonna be using Data Frame Loader. So first of all, if you. For this one, I'm not 100% sure how this one works. I'm gonna also look for a quick example. Oops, There we go. The Lang Chain docs. Oopsie. This one is better. And we just need to create the datacamp loader and then load it. Of course, if you're stuck somewhere, if the instructions are not clear enough for you, what you can always do is as you would do in real life, you can go look it up. But if you're really stuck, and this is something I should have mentioned in the beginning of the project, there's always a solution notebook. In your notebook. But first try to come to the solution yourself. Of course. But if you're really stuck, know that there's always a solution you can look at to see how to solve your problems. Now, for the data frame loader, we have a nice autocomplete here. We're going to say we're going to use the movies database data frame. And as the page content column, we're going to use page content. So this is a column that we just created and that we filtered out here. All of the columns that are not the page content column, they will be added to the document as well as metadata. We'll see that later. Don't forget to call load function. And then we're gonna print the first three documents here. Docs, of course. There we go. All right, so what we end up with is a list of documents. I selected the first three ones here. And you can see the page content is the page content column that we just created. With the unstructured data. And then we also have some metadata, which is right now the source, the IMDb source. So that's pretty convenient. We're finished with the second task, let's go to the third task. And in the third task what we'll do is we'll actually explore the cost of what we're going to be doing next. So as you all know, OpenAI has some costs associated with it and these are typically calculated based on tokens. We're going to use several things from OpenAI and the first thing we're going to do is we're going to use the vector embeddings from OpenAI. And you can see on the pricing page from OpenAI, vector embedding is actually pretty cheap. Like it's 0.0001 token, $1 per 1K tokens. But we want to give get an idea of how much this is going to cost us. Exactly. And for this reason we're going to be using the tick token package. Oops. First of all we're going to create the encoder and that's with the functional function get encoding. And so we'll see, if you go to the pricing page, we can learn about embeddings. So we're going to be using these second generation generation embedding models and what they do. I didn't really mention this yet, but what embedding will do is it will take this information which is unstructured, and it will create a vector from it. So that's basically a list of numbers. And the reason this is interesting is it's really hard for computers to compare strings with each other. So unstructured text to see if they're similar semantically. It is however, way easier to compare numbers or points in an N dimensional space to each other. And what we do when we say we embed text is we basically transform the string into a point into an N dimensional space. And this all sounds very abstract, but the way you can think about this is all of the dimensions in the N dimensional space, they can have a specific meaning. For example, if you would have two dimensions, one dimension could be positive or negative and another dimension could be angry or sad. And so every word, every sentence, every text can actually be transformed to a point on that N dimensional space. Of course, the two dimensions of oversimplification, but that's the basic idea. Luckily, OpenAI has actually trained models on huge amounts of text to have these N dimensional representations of text. And in this case we're going to be using text embedding ADA002, which has 1536 dimension, if I remember correctly. And it's important that we know we are going to use this because for the encoding we'll also use the specific encoding that is mentioned here. So what we'll do next is we're gonna say the tokens per doc and I'll use a list generation here, gonna encode all the documents for doc in docs. And this is gonna give me, actually me, Of course, page content. So token per docs. Of course. What this is going to do is it's going to give me a list with the length and tokens of all of the documents. And this is a pretty long list. I shouldn't have printed this, but that gives you the idea. Now, of course, of course, the total tokens is then the sum of all of the tokens. It's going to remove all of the text here. Where is it? Here it is. So this gives me the total amount of tokens. And now from this we can calculate the cost as the amount of thousand tokens. Because if you remember from the pricing page, it's like this. And we can say the cost per token is.00001. So the cost is. Let me print that out. It's about 37 cents. So that's okay. Okay, so we know what the embedding is going to cost us. So let's get started on using those embeddings. First of all, like I said, we're going to be using Pinecone for this project. So pinecone is what we call a vector database. And what that means is we talked about embeddings. So embeddings are basically the vector representation of an unstructured piece of text. We're going to be wanting to store all of these vectors. So very concretely, what we'll do is these pieces of unstructured text, we'll embed them into the 1536 dimensional space that we talked about before. So we'll embed this, we'll end up with a vector of a specific width, a specific amount of dimensions. Sorry. And then we'll want to store that. And what we can use, we can use vector databases to store these embeddings. And Pinecone is a hosted vector database. That means it actually takes care of all of the infrastructure for you. So what do we need? So at this point, I told you, you already went to the getting started notebook. So you have an account of Pinecone, you have it set up. So the environment variable is set, the API key. So at this point, what we'll do is we'll create an index. So you can compare an index on Pinecone with, for example, a table in SQL. It's basically the thing that holds the entities, and the entities for us are the documents about the movies. So in this task, this is exactly what we're going to do. We're going to be using the Pinecone API, the Pinecone client, sorry, to create an index on Pinecone for the vector embeddings that we're going to get from OpenAI and as I mentioned before, the vector. And this is going to be important right now, vector embedding dimensions that we're going to be using is 1536. And there's actually a source that mentions that here. Like the model name. We're going to be using the encoder that we just used in the previous task and then the amount of output dimensions. All right, let's have a look. I'm going to import os and we need OS because like I said, we're going to be needing that API key. So next we're gonna initialize the Pinecone client and we should do that with. Let me have a moment here. We should be able. There we are. So what we need is we need the API key and I can just use the environment variable there. And what I also need to fill in here is something called environment. And. You'll see in a minute, and I'll show you in the Pinecone console where this is coming from, the environment part of this. So next I'm gonna be listing the indexes or indices, whatever floats your boat. And then finally I'm going to be creating the index. So you'll notice that I've already put some control flow here. The reason I did that is because I don't want to be creating these indexes if they already exist. So there's a small check here to make sure that the index doesn't exist yet. Let me create the. And the name is going to be index name, like we said in the instructions, the distance metric. So we talked about embeddings, we talked about it being vectors, as you know, vectors, they have. They can have a specific distance from each other. The simplest example is like the Euclidean distance. Of course, in this case, what works well often in these kind of semantic searches is the cosine difference. That's basically very simplistically said is the angle that actually is made between two vectors in the N dimensional space. It's related to the angle and the dimensions is going to be 1536. So that should be good. All right, so as you can see, I already ran the cell before. But actually let me quickly, I want to do this properly delete this one. So this printed out the index because it already existed, I deleted it. Now just to show you that this code really. Oh, give me a second. It's deleted. So. Of course there's an error here. So luckily we actually did the double check. The metric is actually cosine. It's not the distance that's cosine and the dimension instead of the dimension. If you are like me and often confuse these things, there is autocomplete to help you with this. Alright, so I removed it, I ran the code and it's there again. As you can see, the metric is cosine, dimension is 1536. Great, let's move on to the next and the most interesting part of this code along. Of course, the later we get in the code along, the more interesting it gets. But this is pretty nice actually. So what we're going to do next is we're going to be uploading the documents that we created before. We're going to be uploading them into the vector database. We have our index, this is what we need to upload that data. And what we're going to do next is we're going to be using LangChain. And in LangChain there's actually a lot of convenient helpers that can help you work with like vector databases like Pinecone or Chroma or other vector databases. First we're gonna be doing some imports. So let's from this one import and that's not how it works. From the vector stores we're gonna import Pinecone and then the last one is from Pinecone. Index, import, index. All right, so as I said, Pine cone is actually to store the vectors. OpenAI is what we're going to be using to create these vector embeddings. So we'll be needing an object called OpenAI embeddings. And if you look this up, you'll see that the default. Actually this is not the. This is the better documentation. I wonder if they mention it here explicitly. Ada. Ah, there we go. So the default is set to be the second generation embedding model that we actually configured or indexed for. So it's always best to check that. But yeah, we can use the default here we create an index object from the index name the IMD Movies index that we did before. Again, I've put in some control flow here. Again, the reason being, I don't want, when you run this notebook multiple times that you keep adding new embeddings to your vector store. There are some limitations there. So just for your sake, I already added an if statement here that basically checks whether there's already vectors in this vector index. So if you run the cell twice, that doesn't matter. It's checking whether there's already vectors in there, and depending on the result, we will be uploading them again or not. In this example, we'll be using from documents, and this is a pretty important one. There's not a lot of things we need to pass to it. So we basically need to pass the index name and we need to pass. Let me think for a moment. Embeddings, of course, so that. Wait a minute, putting this in the wrong one. So if the Vector count is 0, you want to create a new one? Of course. And. Sorry, so I messed up here. Sorry about that. So in this statement, of course, we're checking whether the vector count is greater than zero, and here we're seeing it's greater than zero. That means we already uploaded it. That means we don't need to create the existing. We don't need to create a new. We don't need to upload the documents again because they are already uploaded. We're just going to create this pine cone object from an existing index. And this is actually the other case. So in this case we haven't uploaded them yet. And here we're going to be using the docs again, we're going to be using the embeddings, and the index name equals index name. So what does this mean now? So if there's already documents uploaded, we use the existing index. We don't want to upload them again if there's no documents uploaded yet. What we'll do is from the documents that we created before. So in the docs object, we will be calculating the embeddings, right? And we'll be using the OpenAI embeddings object for that. And then we will be storing it in the index. So there's a lot here that LangChain does for you. You don't need to explicitly call OpenAI first and then call Pinecone, but in the background it's actually doing that. So per 100 documents or so, it's probably going to be sending an API call to OpenAI to get the embeddings, it gets the embeddings back and then immediately uploads them to Pinecone. And what we did here is we called that object Doc Search. So to see that it worked, we're actually going to be performing a search. And in the instructions you'll see that we'll be using a retriever object. So that's an abstraction that's built by LangChain that actually helps you easily get the relevant documents for a specific question. And I forgot the. Yeah, it's trial and error here. All right. And it's running. So because I removed the index before, it actually needs to fill it fill the index with documents, again with the document embeddings. So it will end up in this else condition. And the reason I know this as well is of course because I've deleted the index. But also I can see that this actually takes a long time to run. Now, the reason why it takes some time to run. Give me a second. I have. And actually this is a good demo because you saw it took like 30 seconds to run. If I run this again, the total vector count will actually be greater than zero because we just uploaded some. Because it errored on this. This line right here. And now it will be using the from existing index helper and from an existing index helper. What it will do is it won't upload vector embeddings of documents. It will use the existing index that we just created. Embeddings typo. All right, so the question was, what's a good movie about an epic Viking? And what it sent me back is, first of all, the Northman let check that the source is right. Indeed, this is a Nordman. So it all worked out fine. And the nice thing is we can actually go and look. This is on Pinecone. You can see that it actually uploaded all of the documents with their vectors embeddings. So for example, for this document, for this document, you can see that there's some metadata here. And it also calculated some vectors for that document as it did for all of the other documents. And in total there's about 1500. No, sorry, that's the dimensions. I think it's about 4000 embeddings that we have here. So one of the things to notice here is if you look at the question, the question was, what's a good movie about an epic Viking? And notice that we haven't actually included that information into the embeddings. We included the title, included this description, which is an objective description. It's not a review. And we included the genre, but we didn't include any information on how good this movie is or ratings or like any kind of information that could reflect the quality of the. Of the movie. So asking a question like, what's a good movie about an epic Viking to this system, it's not actually fair because the system doesn't really have that information. So what it's going to do specifically is it's going to take this text and the get relevant documents function here is first going to be calling OpenAI because it's going to create an embedding that means a list of numbers from this question. And it's then going to send a request to Pinecone asking, give me the three closest documents to this vector that I just calculated. And that's going to return like this list of functions of documents. As I said, a good movie, not really a good question. This could be actually something that we also improve the model with. So if we look back, when we calculated. Give me a second. Where is this? If we look back, if we wanted some metric about quality here, we could have included it in the page content that we calculated before. Anyway, let's move on because we're nearing the end. So it's getting more and more excited right now. What we have is the first set, the first part of our RAG pipeline. So we actually have the part where we retrieve the information. We know that's possible. We can. We have the doc search object, we can convert it to a retriever, and we can get the relevant documents next. What we want to do is we want to use this information and we want to include it in a prompt. And the way we're going to do that is we're going to use LangChain and we're going to use the concept of prompt templates. If you've used LangChain before from other projects, this will be very familiar for you. So in this task, I actually already provided some of the prompts for you. So we're going to be looking at them as part of this task. First of all, the imports LangChain prompts. Let's not make too many typos this time. All right? So there's two prompts here, as you can see. And the reason there's two prompts and not one is because we're going to be building a chain. And the first part of the chain is, as I said, retrieving the information. And then we need a way to include that information into the prompt. So the information we received from the retriever is these structured documents. But as you know, a prompt is just a piece of text. So what we're going to have is a document prompt which basically encodes that document into a piece of text. And luckily it's pretty simple right now. So for every document we're going to be displaying the page content first, and then underneath we're going to say IMDb link and then we're going to display the source. You can see that I've been using these kind of delimiters all over the place in the prompt. And this is actually to help OpenAI or the ChatGPT to help it delimit different pieces of context, Much like if you read a text, it's much easier to read that if there's clear sections of the text and so on. So this is the document prompt. And then we'll be using that document prompt to get strings from the given documents that we retrieved. And then we will include it in the question prompt. So this is a pretty long prompt, and it starts with giving the following extracted parts of a movie database and a question. Create a final answer with the amdblink as source. And we say, if you don't know an answer, just say that you don't know. Next, what we do is we give it some an example. And this actually works pretty well. So it actually allows OpenAI to kind of learn from the prompt how it should behave when it answers a question. And this actually also tells you something about how this prompt will be constructed. So there's a question, for example, what's a good movie about the robot to watch with my kid? Then it will include the retrieved information separated by the delimiters, and then a final answer is provided. And in this example, we provide the final answer. It could say something like, the Iron Giant is an animated movie about friendship between a robot and a flow friend and a kid. It would be a good movie to watch with the kids. And it also provided the source of the IMDb movie there. Next we give it our question, and then we have this property called summaries. And summaries will actually be filled in by depending on the retrieved information. So depending on the retrieved information, the summaries will be filled in as well. And then the final answer, this is what the LLM should fill in for us. So we have these strings, and I think they're pretty well as they are right now. What we need to create from them is prompt template objects. I'm going to be creating the document prompt. Oops. And then we have the question prompt, Always with these imports. All right, so that worked. All right, so this brings us with the not the final task but this is the task where we're actually gonna see some nice results. Because what we have right now is we have the retrievers, we have the prompt, we have all the documents where we need them. So we are finally ready to build the actual question answering bot. And what we're going to do now is we're going to use this concept called chains from LangChain. And a chain is as a word suggests. It's basically why we don't just have one AI system that you need to call, but when you want to call a chain of AI systems. For example, in this case, we first want to call OpenAI. So we have a question first. What's a good movie, an epic movie about Vikings? The first part of the chain is we need to call OpenAI to get the embeddings for that question. Then when we have the embeddings for that question, we call Pinecone to ask what's the three nearest documents to this embedded question. It gives back the documents. We create the prompt from the documents and the prompt template that we just saw and we sent that prompt to ChatGPT, which answers, given the context it's provided. And this sounds like a lot of work, but actually we'll see using LangChain, it's not that bad. I'm going to do some copy pasting. Maybe I won't run into syntax errors then this time, Chat OpenAI. All right, so let's see. We're going to be using this chain, I'm going to call it qa with sources. We're going to be using the chain and we're going to be using the from chain type. There's different kinds of chain types. In this case we don't really have time to go over all of this, but in this case we'll be using the stuff type of chain. And that basically means that we first get all the documents and then we basically just for all of the documents, we generate these strings from the document prompt and we will just stuff them in the prompt. An alternative would be, for example, the MapReduce type of chain is where we for all of the documents first send a call to ChatGPT asking to summarize all of these text and then we put the summaries inside of the prompt. But that also costs a little bit more, so we'll just use a stuff chain prompt. Now the LLM is going to be chat OpenAI model name is GPT 3 and a half turbo. And the temperature is going to be zero. That gives us the most predictive results. So we created these prompt templates before we can set these with the chain type quarks, We're gonna be setting prompt question prompt and document prompt to document. All right. And then finally we also need a retriever, of course, because it's a retrieval QA with source chain. So there's a retriever. And we saw before that we can actually do document as retriever. This is actually why I showed the as retriever helper earlier. Alright, so let's see how that works with the question that we asked before. So it's the same question. What's a good movie? A good movie about an epic Viking. Typo? Again, I should have copy pasted. You can see that it takes a short while. And another typo again, should have copy pasted, forgot the O. This is just to build up the suspense, of course. And there we go. What's a good movie about? An epic movie about the dipole Thor, which is epic, of course, and it is kind of about a Viking God. So that makes sense. And it also nicely links to the actual source. So this is what it is, this is what retrieval augmented generation is. And one thing I wanted to stress here is right now this is a movie about 2011, but it could have also answered with a movie from 2022, for example, the Northman. And that's actually a very, very strong characteristic of RAG is it can actually answer with facts that happened after the knowledge cut off of when the LLM stopped training. Another benefit here is that we actually can provide the source, which is very powerful. And as you know, there's a big problem with LLMs about hallucination. And providing a source can actually give some sense of security to the predictions that you make using your LLM. It can give some sense of like assurance that what it's saying is actually well founded. So I have one last task right here. The reason I added this is because as we said, LangChain actually provides a lot of things out of the box, but it also abstracts away a lot of the technicalities. And I wanted to show how you can use one of the features of LangChain, namely the debugging mode, to get some more insight into what LangChain is actually doing. And we're going to be setting LangChain debug to true. And I'll just be repeating the same question again. There's a lot of output now, way more than we had before. And I'm going to scroll through this a little bit. So what you can see here is actually what these chains exactly do. So we have a chain that starts the first one. It's a retrieval QA with sources chain. And the only input it has is what's a good movie about epic Vikings. Then what it's gonna do next is it's gonna have some like there's a lot of output here. But the main thing here is the next chain that's called is the LLM chain. And we pass through the stuff documents chain as well. So what happened in between is that this piece of text has actually been sent to OpenAI to get the embeddings. As we said before, we retrieved some of the documents. And next we enter the LLM chain with the summaries of the documents. And these summaries, these are actually created from the document prompt that we saw before and the documents that were retrieved from the vector database Pine cone in this case. What's next is from these question and summaries. As you saw in the prompt template before, we have question and summaries, we create the actual question prompt, given the following extracted parts, and so on and so on. And this will actually have the information about the Viking movie in there as well instead of the summaries property that's Then sent to OpenAI. OpenAI and the response is tor is a good movie. And so on and so on. And it puts the source in here, it's all text. And then we go back up the chain, through the stuffed documents chain to the retrieval QA with sources chain. And this will actually parse this text, this source into an extra field and it will provide the answer field, which is a text before source. And then it will also provide the source itself. And that's it. We've actually done some retrieval augmented generation. I hope you've learned from this. I hope there was nothing. Too many typos. Probably each cell had one. But that's what happens when you do some live coding. So yeah, I hope you learned from this. I hope you can apply it in your real life. Some good ideas for improvements is one we already discussed is probably going to be to add more information to the embeddings. So instead of the title, genre and description, we can also add some quality metrics, maybe even a review or two. That's up to you. Another good addition to this would be to build a more conversational bot that actually can answer some more questions. So for example, if you ask it what's a good movie about Vikings, it could ask you first, should it be for kids or for adults? And so on and so on. So that would also be a good addition to the system. But of course, it's up to you. I hope you have fun with this. I hope you had a lot of fun and keep learning. Bye. Bye.