RAG with Llama-Index: Vector Stores

This is the third video in our series on Llama Index where we are building an end to end chat with your document system. Today we are going to be focusing on vector stores. Vector stores are an essential components of any RAC system and they actually determine how the user can interact with your system. So primarily there are two different types of vector stores. One are self hosted which you can host on your local servers or local infrastructure. The second one is cloud based solutions such as Pinecone. As the name suggests, the vector store is going to be stored in the cloud. Now, depending on your intended application, you are either looking for a self hosted solution or a cloud based solution. So in this video I will show you one self hosted solution. So we're going to be looking at Chroma db. This is an option that you can run on your local servers or your local infrastructure without access to the Internet. But if you are serving clients or users or the Internet, you probably want to look at a cloud hosted solution such as finecone. So we are going to be covering both Chroma DB as well as Fine cone in this video and see how you can interface both of them with Llama Index. Okay, so I put together this Google Colab that I'll share with you. Now in order to run this, you don't really need a GPU instance. So in this case we are going to be using an open source embedding model. And for the LLM we're going to be using OpenAI's LLM. In a subsequent video I'll show you how to actually replace the LLM part with an open source model. Okay, so first we are installing everything that we need. So we will need to uncomment this and run these two cells. Next we are setting up our OpenAI API key. This part is needed to access GPT 3.5. Next we are importing different packages that we will need and I'll talk about each one of them when the time comes. Now for this specific use case I'm using the BGE Base English model for computing embeddings. So this is available on Hugging Face. You can replace this with any other open source embedding model or if you would like you can Even use the OpenAI's embedding model. Now in order to do chat we will need some documents. So in this case I'm using the Orca paper and the way I'm doing is I'm using the Simple Directory reader to load that file. Now I have uploaded that file in here in the Google Colab and the way you do it is you simply right click and then click Upload, Select a file and that will be uploaded in here. Or you can create a folder and upload files to that folder. But in that case, you want to provide the folder name in here instead of. So you replace this with the folder name rather than a single file name. Right, but in this case, we're just dealing with a single file. That's why I'm using this Input files key. Right. If you are uploading multiple files and read them by name, you can provide them in the form of a list. Now, I went over the basics of Llama Index in my earlier two videos. I'll put a link to the playlist. So if you're new here, I would recommend watch those videos because that will give you a better understanding of what this simple directory reader is. Okay, so we loaded our document. Now let's see how we create Vector Store within Llama Index. So the first one we are going to be looking at is the Chroma db. Now, the way it works is we first need to create a Chroma DB client. So here we are creating that, and then we need to create a collection that we are going to be storing in our vector store. So, so for that you can assign it a name. I simply called it Orca Paper. So basically, the client that you created, you create a collection with your name of choice. I'm simply calling it Orca Paper. So we just created a new collection. Now, based on this collection, we need to create a vector store. So for that we are going to be using the Chroma Vector Store class. We pass on the Chroma collection that we just created and we assign it to a variable called Vector Store. Now, using this approach, it really gives you the flexibility that you can have multiple different collections and multiple different vector stores if you are dealing with a diverse set of data. Next, we simply need to change the default vector store that Llama Index is using. So the vector store we just created using Chroma db, we assign that and we're using the storage context and changing the default value. Right? So now our storage context is going to be updated with this new vector store that we just created. Okay, we also need to change one more default value, and that is the embedding model. So by default, llama index uses OpenAI's embedding model. Now, in this case, since we wanted to use the open source embedding model, that's why we're going to assign or change the default embedding model in here. Okay, so if you go up here is where we define our embedding model. Okay, so we are all set with our defaults. Now we simply need to create a vector store index. And here is how you do it. So we're using the vector store index class and we're creating the vector store from the documents. So this is basically the paper that we provided. And we need to change both the storage context. So now we're going to be using the Chroma DB vector store and we also need to change the service contacts. So in this case we only change the embedding model. So this will update the embedding model as well. Now, in terms of our architecture diagram, we are actually updating this part. So we are updating the embedding model and the vector store where we are storing both the embeddings as well as the text chunks. Now, how do you use it? So from the index we start a query engine that becomes a query engine and then we ask a query, right? So, for example, here the question is, what are some of the main contributions of this new ORCA model? And it gives us a response that we are displaying in here. So I'm not really concerned about the accuracy of the response at the moment. I just wanted to show you how to set up a Chroma DB vector store and use it with Llama index. Now, by default, when you create a Chroma DB vector store within Llama index, so it will remain in memory, we are not writing anything to the disk. But if you want this in real applications, you want to create your vector store and then write it to a disk and read it whenever you need it. So here is how you do it. So first you need to create a db and this time we are using the persistent client. So it's going to be written to this specific path or folder. Now we are we need to assign it a name. So I simply call it DB Collection, right? So that's your collection. Now after that again we will create the vector store, change the default vector store so that the vector store index can use it. Now here for an example, I'm again creating the service contacts. In this case, again, I'm changing the default embedding model and also setting chunk size, overlap size. So these are different parameters that you can set within the service context and you can simply pass those on to your index. Now, when you run this, so it will create this new folder called ChromeDB and there's going to be a collection called DB Collection. Now, if you have a Vector DB, a Chroma Vector DB that you stored on the disk. In order to load that, you're going to essentially follow the same steps. So we are creating another DB called DB2, right? Creating a collection. Now you want to use exactly the same name that you used for storing the db, right? And after that the rest of the process remains similar. So you create a vector store. You pass that vector store to the vector store index right along with the service context. If you have modified anything in here and you can start interacting with the index there, you just load it from the disk. Okay, so we looked at a self hosted option. Now let's look at a cloud based option. So for this example we are going to be looking at Pinecone. Now you need to install the Pinecone client for Python and we are simply importing that client in here. Okay. First we need to set up the Pinecone credentials. So we need both the API key as well as the Pinecone environment. Now keep in mind, Pinecone is not free. They do offer a free tier where you can create a single index. So that's what I'm going to be using. But if you're serving your application, it's actually good to get the paid plan. So you will need to create a Pinecone account. I'll put a link in the description, then go to the API keys. So here you will find the environment. So simply copy that and paste it in here. So this is the GCP starter. And then in order to copy the API key, you can simply click on this icon and this will copy the API key. So after that you will paste your API key in here. Now the initial setup in here is a bit more involved. So first we need to initialize our Pinecone instance with the credential that we have in here. So we're going to call pinecone.in it and then provide both our API key as well as the environment that we are working with. Okay, Next we need to create our index. So we are calling it Llama Index, Right? And first we are checking if this exists. So if it doesn't does not exist, then I'm creating this index. So the name is going to be Llama Index. You can call whatever you want. Now here are the two most important things that you want to actually be careful about. First is you want to define the dimension of your vector store embeddings. So each embedding model comes up with different dimensions. So you want to be careful and you need to set those dimension correctly in here. And the second part is the distance metric. So here I'm using the cosine similarity, but you can use something like dot product as well. Right? So there are multiple options in here. Cosine seems to be the fastest one. Now where do you get the dimension of the embedding vector that or the embedding model that you're using. So if you are using open source embeddings, so Hugging Face has this embedding leaderboard, you can go in here. And for example, I'm using this specific model, BGE base English version 1.5. So here the embedding dimensions are 7. 68. I believe for OpenAI models it's 1534. But make sure that whatever model that you use, you check the dimension of the embedding vector and simply assign it in here. Next you need to set up the pine cone index. So here is how I'm doing it right? And then I'll just print the parameters. So for example, you can see we're using the cosine similarity and the dimension is going to be 768. Right? There are some other information related to the vector store that we just created. Now, there is absolutely nothing in the vector store, but this process will simply create an instance on Pinecone website. So if you go here to Pinecone index indices, so you can actually see that it created a new index and it's called Llama Index. Now I'm on the free tier, but if you have paid tier, you will be able to create multiple of these. Okay, so through this process we are only done with the Pinecone side. Now we need to bring this index and connect it to Llama Index. Now the way you do it is you're going to be using a special package within Llama index and it's called Pinecone Vector Store. Now this is very similar to what we did for ChromaDB. So first we need to simply import the storage context. Right? We're creating a vector store based on the Pinecone vector store that we created. So we simply assign that index in here. Then we need to again update the storage context defaults. So here I'm replacing the default vector store with the newly created vector store here and I am again actually updating the embedding model, although I think I don't need to do it in here because we already updated it in the previous section. Next we need to create a vector store index based on all the parameters. So I provided the document, I provided the updated storage context as well as the service context, and this will create a new index for us. So this process is going to take some time, depending on how many documents you have in there, right? For example, I think it created 55 different nodes and it had to also update it or uploaded to Pinecone server. And now we can simply set this as a query engine and start asking questions from that index. So in this case, the responses that you are getting are based on the vector store stored inside Pinecone. So it's not a locally hosted vector store anymore, but we are hosting this on Pinecone's servers. Okay, so let me summarize this in three different steps. First and foremost, you simply need to create your vector store. So depending on whether you're Chroma DB or you're using Pinecone, you're going to assign it a name and initialize it. Then you need to create a vector store object that is going to be used to change the default value in the storage context. And then simply using this new vector store, simply you need to compute all the embeddings from your document and store them as a part of the index. If there is interest in Llama Index, I will continue making videos in this series. Let me know in the comment section if that's something you will be interested in. If you would like to support my channel work, check out the YouTube member community by clicking the join button. There is going to be exclusive members only content and perks. Now, if you would like to learn more about Llama Index, check out this playlist. It has all the videos. As always, thanks for watching and see you in the next one.