Talk to YOUR DATA without OpenAI APIs: LangChain

A lot of you have been asking how to do information retrieval in LangChain without using OpenAI. So I thought it would be nice to make this video in showing how to use other open source models with LangChain. So in this video I'm going to show you how to do information retrieval from text and PDF files without using OpenAI's embeddings and and without using OpenAI Large Language Model. In this video we will be looking at two applications. First we will deal with text documents and introduce different concepts and models. And then we will look at how to work with multiple PDF files. So first we need to install some of the required packages. In this case we are installing LangChain, Hugging Face Hub and Sentence transformers. Next we need Hugging Face Hub API token since we are going to be using models from Hugging Face. In order to get the token, go to your Hugging Face account, click Settings and then under settings you will have these access tokens. You can create a new token or use an existing one if you already have one. Side note, always make sure if you're sharing Google Colab notebooks, always remove your tokens. In one of my previous videos I forgot to remove my OpenAI API token and it was a costly mistake. Next we need some data to work with. So in this case I am just using State of the Union address, but that is an example provided within LangChain library. So what we are doing it here is we are importing the request library. So here is the URL of the text file that we are going to read. Then based on the URL we use request library to get the text file and we write it to a File name stateoftheunion.txt. when I execute this cell, the file will be stored on my Google Drive. Next we are using the text loader function from documentsloader to load our text document. Now if you run this, we can actually look at the content of our document. Now we have our document. Next we want to divide it into smaller chunks so that we can feed this into our large language models or embeddings. Each model has a specific token size. So in this case we are choosing a token size of thousand and you want to make sure that it doesn't exceed the token limit of the model that you're going to be what you will. So we're using character text splitter. This will convert a document into chunks of thousand tokens and those are going to be stored in this documents object. There are a total of 42 chunks and here is the first chunk. Next we want to compute embeddings from our documents. So in this case we're using embeddings from hugging face instead of OpenAI's embeddings. Now, this is not the only type of open source embeddings that LangChain supports. There are actually a number of different embeddings that you can use. So here is a list of text embedding models. You have the OpenAI that is UNLMM CPP cohere, you can use fake embeddings and there is instruct embeddings as well. So have a look at these. Depending on the application that you are working on, you want to select specific type of embeddings. Keep in mind that the best embeddings currently available are open AIs embeddings, but those are not free. So in this case I'm simply creating an object of embeddings. Right now we need a vector store for information retrieval. So in this case we are using this vice CPU and so you just need to install it. I think I already have it. Okay. There are other vector stores as available in LinkedIn. You can check out a comprehensive list here. Next, using these embeddings, we need to embed our document. So in this case we're using from document pass on our documents or the chunks that we created, right. And the corresponding embeddings. And this variable is now going to have embedded document. So let's run this. It's essentially a database, a vector store of the embeddings from our document. I have a very detailed video on the topic, so I'm going to put a link to that video if you're interested. Watch that out for more comprehensive coverage. Now, in order to do a query on our documents. So we get the query, calculates its embeddings and then do a similarity search between the query and the document's embeddings. So for example, in this case, the query is what did the President say about the Supreme Court? Using similarity search, it will find the documents that had the text closest to this text. So it came up with a few answers. So for example, here, Justice Stephen Breer talking about the United States Supreme Court. Then again here, Right. So there are a couple of places based on the similarity search it found. Now, these results are solely based on the similarity search. Now you can also integrate this with a large language model to do a Q and A based on your text files. Next, I want to show you how you can integrate this as a part of question answer chain Using a large language model. So in this case we are loading the question answer chain from question answering. Then we're using the hugging face hub and the large language model instead of OpenAI's model we are using is the Google 95 Excel. Right. So it will get the large language model from hugging face. Next we need to create a chain. So in this case instead of OpenAI LLM, I'm going to pass on this LLM that I created and let's run this. So we have our chain in order to run a query or a prompt, here is what we do. So we have a query, then find, then do the similarity search on the documents based on the embeddings that we provided and then the change will simply change the creating and the document in paddings together to get a response. Let's run this. All right, and here is a response based on the information that we provided. So one of the most serious constitutional responsibility of a president has is nominating someone to serve on the United States Supreme Court. Now this is basically a question answered bot that you can create. Here is just another prompt that I ran using the same chain. So what did the President say about economy? He said the responses build the economy from the bottom up and middle out, not from the top down. All right, just keep in mind that the responses may not be really accurate because the model that we are using this is not as good as OpenAI's models, but it's free and you can experiment with it. Next, I wanted to show you how you can do the same with multiple PDF files. This is going to be a little different than what I had in another video, but I think it will be something interesting. We're going to be using the same basic concepts that we just learned. So first you need to install the required packages. Next we import two different functions. So unstructured PDF loader and Vector Store index created. This is the function where all the magic is going to happen. Next, we are simply connecting Google Drive to this Google Colab notebook. So I have a folder within my Google Drive called Data underscore 2 and it has two different PDF files. The first is this paper on or the technical report on GPT for all. And then there's another paper related to Lib syncing. Next we are creating loaders to load our files. Now since there are two PDF files, the result will be list of two rotors. Next we will look at how do we create Vector store. So this function Vector Store index creator accepts embeddings. In this case we are passing on the hugging face embeddings that we had already earlier defined. And then we are dividing our documents into different chunks, right? So a chunk size of thousand and then pass on the loaders that we had. So this will basically has embeddings created from these documents. Now if you run this, you will probably see this warning. Daktron 2 is not installed. That's absolutely fine, you can simply ignore it. It simply affects the speed. But I have found that if I try to install this it throws an error. So I'm going to ignore this. Next we need our large language model that we are going to be using for question answer. So in this case again I'm using the hugging face hub and getting Flan T5 model from Google instead of OpenAI's DaVinci models. For our information retrieval we are using retrieval Q and a chain. So in this case we pass on the large language model that we just created. Chain type is stuff and then the retriever is the index store that we created. By default it's using Chroma db. We could actually change this, but I think I left it to default. So instead of files it's going to be using chroma DB and then the input key is going to be question. So that's the prompt or the query that we're going to be passing it on. All right, so let's run this. So now we are ready to run our query. So that the way you do it is you have chain dot front and you pass in your query. So let me ask it, how was the GPT4 onward train and says using Lora. Now let's run another one. So let's see who are the authors of GPT for all technical report. Okay, Yarish and I think it got only the first author. You can play around with the maximum length and stuff so you get better responses. So there you have it. I just wanted to make this quick tutorial on how you can use open source models instead of OpenAI's models if you don't want. Okay. Hope you found this helpful. If you have any questions or comments, please put them in the comment section. I would love to answer them if I can. Thanks for watching. See you in the next one.