ChatGPT for YOUR OWN PDF files with LangChain

If you have a bunch of PDF files and you want to have a conversation with them, just like you can have a conversation with ChatGPT about any topic, then this video is for you. I'm going to show you a website which uses the exact same concept and does the exact same work. I'll also show you how to get free credits from OpenAI so that you can experiment with this. And I'll also show you my usage and the cost associated with it. For our application, we're going to be using Langchan, which is a framework for developing applications powered by language models. It allows you to connect your language model APIs to other sources of data, and then it allows the language model to interact with its environment. So in this case, the data sources. You can use this with any model from OpenAI, along with all the recently open source models such as Llama, Alpaca or GPT for all. Before looking at the code on how to do this, let's walk through this architecture diagram to have a better understanding. And don't worry if you are not a code. I will walk you through each and every line of code and explain what's going on in there. It's going to be pretty easy to follow, so let's get started. So we start with PDF files. You can have multiple PDF files. For this example, I'm going to be just using one PDF file. Then you simply read that PDF file and extract data from it using Python. Now, say you have 100 or 200 pages in your document. You cannot really feed that into your large language model because it will hit the token limit. So in this case, what we do is we divide it into smaller chunks, such that the length of the chunk is smaller than the token size that the model supports. For our example, we are dividing our document into 10 different chunks. Next, we convert each 10 chunks into their corresponding embeddings. An embedding is a vector or list of floating point numbers. And essentially it works as a compression algorithm. Let's say each text chunk has thousand characters. But using embeddings, we can use it reduce it to a much smaller size. Let's say the embedding size is only three, right? So that will be the compression which is performed here. In this case, we are going to be using OpenAI's text embeddings. And now with the help of embeddings, rather than comparing text directly, you can simply compare embeddings and see which two different texts are closer to each other. Based on these embeddings, we simply create Our knowledge base. Now, with this new knowledge base, when a user asks a question, we will use OpenAI's embeddings to convert that text into embeddings. So it's going to be a list of numbers and then we'll use a vector database to actually run a query on the knowledge database. So based on the embeddings of the documents that we have stored, we will get results and they will be ranked based off their closeness or relatedness to our query. So we'll get the results here and then we will use a generative large language model to generate a response rendered back to the user. So I hope this process makes it simpler in order to understand what is going on. Now let's look at the code. Here is a notebook that I put together for this exercise. So first you actually need to go to the runtime and in this case you don't really need a GPU runtime. So the normal CPU runtime is fine, right? Then you need and come here you will see a connect button. So click on that. If you want to run a self, you can say this. See this play button with it. So if you click on this, it will run the code cell running. The first cell will install all the packages that we need. For this specific example, I'm using this as a technical report of GPT4 for all. I have a video on the actual GPT4 model. So I'm going to put a link to that if you are interested in watching. So it's basically a PDF file for their technical report. And we are going to be using this and asking questions from this next cell. If you run this, it will load all the packages. Next we will run this cell where we need the API key. So since we're using OpenAI, we will need the OpenAI API key. Now in order to get your API key you can go to this link. If you go to the link you will be asked to create an account. I already have the account, so I don't need to create one. Then you will see something like this. API keys. Click on this and then you won't have any API key. So it will ask you to actually create a new one. So let's say this is the API key, right? You can just copy it. Close this. I am going to delete it afterwards. So don't try to use this specific API key. But I just wanted to show you the process. So after that simply paste the API key and now let's run this. So since we are running this in on Google Colab and my file is actually in my Google Drive. So we were going to connect our Google Drive to the Google Colab. So run this. It will ask you for permission to connect your Google Drive. Say yes and then simply allow it. And that's all. Then you will see this message saying that your Google Drive is mounted. Okay. And this is the base address for your Google Drive. Okay. Next, I have the technical file in this folder on my Google Drive called Data. So I simply put in the pathway of that folder. Right? And we are going to read that file. So you just run this and it will create this reader object. Right? So if you run it, this is a reader object. It has all the information on how to read the contents of my PDF file. Then we want. But we want the raw text, Right? So if you run this cell, it will actually go to each page and render. Read the text from each page. Right? And we'll simply return the raw text. So if I type in raw underscore text, these are the content of my PDF file. And it didn't really show the whole file because of the space limits, but it simply truncates it there. Okay. All right, so we have a raw file. So now we need to divide it into different chunks, as I showed you in the architecture diagram. And we are using a chunk size of thousand characters. So that will ensure that we are not hitting the token size. And in this case, I'm using an overlap of 200. So that that would mean that we get the first sentence will have the thousand characters, and then the second sentence will start from the 800 character and onward. All right, so let me show you what I mean. So let's say that's it will give us a total of eight chunks. Right. And if I read the first chunk, then. Sorry. Okay, let me run this. That's the text in the first chunk, but if we add another chunk to it, so let's say text the second chunk. Now it starts from. We collected roughly 1 million prompt responses, and there is an overlap. So if you see here, we collected roughly 1 million prompt responses. Right? So this is because we define overlap of 200 characters. You don't have to do it, but in my case, it seems to be helping. Okay, next we download embeddings from the OpenAI. As I said, embeddings is simply a list of numbers, a float important numbers. Right? And you can use this to measure the distance between two different text strings or sentences. So here we simply downloaded the embeddings from the OpenAI. But now we want to convert or actually find the embeddings of our text chunks. So for that we are using this vector database. So essentially what's going to happen is it will take the text chunks and finding their corresponding embeddings, and that will be stored here in the document search. Next, we are importing question answer chains from LangChain and the corresponding OpenAI object. Now here you can actually pass on different models that you want. For example, by default I'm using this taxadder001 model. But if you need a more powerful model so you can use those, you can, I think even use GPT4 API calls if you have it. But just for this simple experiment, I'm using this model. Okay? So this will create a chain, right? And now we are all set to start asking questions. So for example, here my first query is, who are the authors of this article? Right. Then from our embeddings we simply do a query, right? So we'll give this text and it will find out which is the closest text in the document semantically to our query search, right? And run the chain on it and give us a response. So let's wait for it. Now the response is Yanovish Anand Zach. I don't know how to pronounce the name, but we can actually go to the article and see who are the authors, right? So you can see that's the first author, that's the second author, and then third, fourth and fifth author. So it was able to actually correctly and you retrieve that information from the document. Now the beautiful part is like there is no list of authors. Like let's say author one, author two, author three. But even from the way the technical report is written, it's able to actually infer that these are the authors. Because if you go down, there are other names of people as well. But it completely ignored those. So that's pretty awesome. Okay, next I'm asking what was the cost of training GPT4 for all model? Right? And it returned an answer of $100. Right. If you go back to the article. So here it says that our release model can be trained about on about eight hours. And here's the total cost. Now the interesting part is there is another dollar figure like 500 and then even 800. But it still was able to actually get us the correct $4 figure because these are the cost of training other models. It's pretty neat. Okay, so I asked it a couple of other questions. For example, how was the model trained? So the model was trained with Laura on it. Gave us actually then how many training examples or data set that was used for. And that's pretty good. And then how was the size, what was the size of actually should be what was the size of the training data? Let me change this. Although like it said, give us a correct answer. So this many prompt generation pairs were used to in the creation of the data set. Another question, how is this different from the other model? So it kind of gave me a whole description. It says that it is non commercial, it has non commercial license, it is trained with Laura. Right. So I think it's not really an exact answer because that exact information is not really present in the paper. So let's ask is something that is not in the technical report. I will say what is Google Bard? Okay, let's see what the response is. And the answer is I don't know because that information is actually not in the technical report. Okay, so it's very powerful. Now let me show you a website that is actually using the same concept to let you query your own PDF documents. This website is called PDF TPTIO and it used to be actually free, but now it requires you to give you to give them your API key and then you will be able to query your documents. But anyways, here's how the interface looks like. So you simply upload your own PDF file, right? And then you can simply start asking your question and it will start giving you responses. So it's very similar to what we did in this tutorial, but they have a pretty nice interface around it and they are using a little bit more powerful model than what we had. Now let me also show you the usage cost that we had with this tutorial. If you go to your OpenAI's account using the link that I provided. So first as I said, there are API keys that you can see, then you can actually go to the usage, right? So they give me a credit $5. So far I have used around 50 cents of that and I have made around 62 requests. So it's. It's a minimal fee. It's not that crazy, but something to be careful about. Well, I hope you found this video useful. If you did, don't forget to comment. And like the video, it really helps with the algorithm. Thanks for watching. See you in the next one.