CHATGPT For WEBSITES: Custom ChatBOT: LangChain Tutorial

In this video I'm going to show you how to create custom chatbots for your websites. I'm going to walk you through a step by step process to do this. Usually websites are composed of multiple pages. You can list all those pages using the sitemap of a website. For example, Here is the OpenAI's website, which is composed of multiple links or web pages. And you can actually navigate to another page by simply clicking on it. However, if you want to list all all of the pages in this website, you can simply use the sitemap. Now I Simply added/sitemap XML and if you access this website or this page, you will get a list of all the pages that are in this website as well as their corresponding addresses. So These are the URLs in this video I am going to show you how can you use these individuals URLs as a source of information for your chatbot. And the chatbot is going to instruct information from multiple URLs in order to create a response for your visitor. So first we will look at the overall architectural diagram for this chatbot and then we will look at the code implementation. So let's get started. Okay, so as I said, the site map keeps track of all the web pages. They are linked to your website. So let's say if you access the sitemap, this is going to be connected to different pages. So let's say you have page 1, 2, 3 and this is page N. So depending on the website there are going to be different number of pages which is represented by N in this case. Now in order to train a chatbot, we are going to be using a large language model. So it can be either through OpenAI or any of the open source large language models out there. In our case, we will take a web page and then divide it into smaller documents. Now the reason of doing this is that we're going to be feeding this into a large language model. And the current large language models that we have, they have a finite context length. That means it can process only a small fixed amount of text. So because of that are converting each page into several documents. So for example, page 1 is converted to n different documents, right? Then page 2 is converted to another n documents and page 3 is converted into another N documents. Okay, I just fixed a typo. So this is the second document. So that's 2A to 2N. Now when I'm going to show you the code, you're going to see something called chunk size. So this is basically how many parts or documents we create. And each document is going to be a specific length that is going to be defined by the chunk size. Next, we convert. We need to convert our documents into embeddings. Now, you may ask, what is an embedding? Now, in order to explain the concept of embedding, suppose that each of the documents that we just created has a chunk size of thousand tokens. Now, embeddings are used to compress this into smaller vectors containing numerical values. So by using embeddings, you might be able to convert these thousand tokens into much smaller vectors. For example, it's going to be float numbers. So it might look like something like probably this, right? And let's say using an embedding vector, we might be able to reduce it into a vector of size 3. This is just an example. And the real beauty of embeddings is that comparison between different documents or your query is actually a lot easier in the embedding space than the original text or token space. Now, here you see that we converted each one of the documents into its corresponding embeddings, right? And then we simply store all these embeddings into a semantic index and that becomes our knowledge base. So this is basically all the knowledge that you have on your website. In this specific case, we are going to be looking at FAISS Index Store. FAIS is a vector store created by Facebook Research. It is a library for efficient similarity search and clustering of dense vectors. Now, FIAS is not the only option. There are some other options. For example, Pinecone, Chroma db. Right, so these are different vector stores and depending on your need, you can select any one of them. Now, so far we only talked about how you create a knowledge base from your website. Now, one thing to remember is if you add a new website, for example, if you have a blog and you add another blog post, right? All you need to do is simply compute embeddings for this new blog post and append it to your knowledge base. You don't really need to redo it for everything else. Okay, so we know how to create a knowledge base, but what happens if a user is interacting with a chatbot? Right, so this is. These are different steps that we're going to be following here. In this case, what happens is if a user is interacting with the chatbot, the user asks a question, then that question will be sent to the embedding API. And depending on what API embedding API you used, for example, you can use OpenAI's embeddings or some other embeddings, for example, GPT4 or Llama index, right? The same embeddings are going to be used to embed your question or query. Now let's assume that turned out to be this is probably, let's say this is the embedding that you get from your question, and these are the embeddings that you have in your knowledge base based on the documents that you have right now. Next step is through a semantic search or similarity search. So what you want to do is you basically are going to take the embeddings of your question and compare it with the embeddings of the documents that you have already stored in your knowledge base, right? So whatever similarity metric that you use, you will get different ranking. For example, you want to only get the top four similar documents, right? So it will simply return you the embeddings of those four documents. Since we know the indices of the corresponding documents based on the embeddings, you can actually retrieve the actual documents, right? And those documents basically becomes your context. Now this is essentially the text on which you want to perform your query, right? So next what's going to happen is you would take the documents that is your context, right? And you simply take the original question and feed these two information into your large language model and you will get an answer. So the large language model is going to look at the documents that the similarity search found to be similar to the question based on the embeddings, right? So use that as a context to generate answer for your question, right? And as you use it, you get the response back. Now just to reiterate, there are two main components of our chatbot. First is the embedding, so that is simply used for finding the most similar documents. And the second component is the LLM for generating the response in natural language. Now, depending on your application, you can actually choose any other type of embedding that you want. And the LLM can be any LLM. It can be based on OpenAI's large language models, or it could be one of the open source large language models. You can actually mix and match them as well. So for example, you can use OpenAI's embeddings because let's say they are really good for document retrieval, but you can use another large language model, let's assume something like GPT4, all for generating the natural language response. So I hope this explanation is clear and it explains the theoretical concepts. Next we are going to look at the code implementation for this chatbot. Now let's look at the code base of how we can implement this. I am running this on a local machine. In this specific example, I'm going to be using OpenAI both for the computing, the embeddings as well as for the large language model. However, I have a detailed video on how to do this using open source tools. I'm going to put a link to that video as well as I'm going to be creating a completely new video using GPT4All as your language model in llama indices as embeddings. So keep an eye out for that video. In the first block, we are simply installing the required packages. In this case, I'm using the FAIS vector store, so the CPU version, LangChain and OpenAI. Next you want to set your OpenAI key. You can find this in your own account. All right. Now, in order to get your Open A API key, simply go to your account. There is this section of API keys. Then simply click create a new API key, give it a name, then create the API key and just copy that key. Next, you want to get a list of URLs, right? If you have your own website, you can use the website sitemap to get a list of all the URLs. In this case, I'm manually providing a list of three different URLs. This is just for test purposes, so I don't really want you to go and scrap all the pages on a website. I went to the announcement of three large language models. So one is the MPT7B, the other one is the stableLM, and the last one is Vicunia. Right? So actually went to these pages, copied those links here. This is going to be essentially the database that I want to interact with. For example, here is the blog post that was released when they were announcing Vicuna, right? And similarly there are blog posts for Mosaic ML MTP7B models as well as for stablename. Now, the reason I chose different website was I want to also show you how you can retrieve the source of information from your model. Next, you want to read data from these URLs, right? So for that we are going to be using the unstructured URL loader from Document Loaders in LangChain, right? So what you need to do is you simply need to pass the list of URLs to your unstructured URL loader and then call the load function on your loader. This will give you all the data comprised in your URLs. Right? So for example, here I looked at it and these are essentially three different web pages, right? So for each web page you have a different loader. And this shows like the whole text contained in each one of these web pages. Based on our architecture diagram so far we are just on this step two, right? So assuming you have a site map or like a list of URLs, we just got here. Next, we need to divide each of these web pages into smaller documents so that we can feed this into our large language model as well as the embedding computation model. So for that we're going to be using the character text splitter from text splitter in LangChain. Right. Now what is going on here? And I want to explain the design choices here. So first and foremost, we want to divide each one of the documents or web pages into a token size of thousand. Now, roughly speaking, think of a token as a word. So first thousand words into one chunk. Then we are defining an overlap of 200 tokens. Now, there are cases in which overlap is very important to have. For example, if you are conveying an idea in multiple sentences and the last sentence is dependent on, let's say, the previous sentences, right? If you simply do a normal chunking without overlap and the last couple of sentences end up in the separate document, then in that case it may not be conveying the information correctly. That's why you need to have an overlap so that there is continuity of information. So in cases where you have temporal dependence of sentences or sequences, you want to use an overlap. In the cases in which there's no temporal dependence, you can ignore overlaps. I like to keep it to around 10% or 20%. Now, I hope this is clear. Now, the last design choice is the separator, which where we are using new lines. So the chunk will end at a new line and it will be divided based on the next sentence. Then what we do is we take our data so all these three documents, right? And we run it through this character text split running. So as a result, we are going to get a number of documents, right? So you see here instead of just three, we actually will get a total of. Let me see, how many documents do we have? Okay, so now we have around 62 documents. And this is what you get, right? So we define like N documents. Now, each one of the page is not going to give you the exactly same number of documents documents because different pages may context contain different amount of text. Okay, now if you have seen my previous videos on information retrieval, the rest of the process is very similar. But I want to Go into more details of the models themselves, right? So, so the next step is to get embeddings. And for this case we are using embeddings from OpenAI, right? So even for embeddings, there are a number of different models available within OpenAI. So for this specific case, we are using the default one, which is the text embedding ADA002. Using that API, we simply compute embeddings for each of the documents that we created, right? But next we need to store them in a vector store. So for that purpose we are going to be using the FAIS vector store. So what we do is we take our documents and our embedding model and based on that, we're going to compute all the embeddings and store them in the vector store. I usually write them to the disk so that I don't have to recompute these embeddings again and again because there is an associated cost with it, right? So all you need to do is just dump the embeddings on your hard disk and then whenever you need those embeddings, you simply read them from the disk rather than computing them again. Now the next step is to do information retrieval. So that's this whole step of somebody asking a question from your chatbot and interacting with your website. Now, in order to do that, we're going to be using LangChain's retrieval question answer with sources chain. Now, I'm using this specific chain because apart from the answer, I also want to see where the answer is coming from. Now, in order to generate responses in natural language, we need a language model, a large language model. In this case, we are using a default model from OpenAI. So let's look at this. The default model is the text DAVINCI003. Now you can also look at some of the parameters we set the temperature, the maximum token is 256, right? But we can actually change the default models. So let me show you how you can do that. So if you go to OpenAI's website, there are a number of models that you can use, right? So even you have the ChatGPT or 3 point GPT 3.5 turbo. Right? Now, in order to change the model, you will need to pass another parameter to this function called model name, right? And then you simply define what type of model you want to use. I'm going to keep it to default, but just wanted to show you this is where you can simply change the model name. Okay, so we have our model, we have our Vector store. Now we need to create the change. So in order to create this retrieval change chain, we're going to be calling this function from LLM, right? So that's large language model. Then you pass the large language model that you're using. Again, it doesn't have to be OpenAI, it could be any other model. And in the subsequent video I'm going to show you how to do this whole process using absolutely open source models running on your local machine without hugging face. So hugging face is another option which you can use, but I'm going to show you a completely different process. I already have a video on hugging face. I'm going to put a link to that here. Okay, so we pass on the language model and our vector store, right? This and we create a chain. Now next, when we pass a question to our chain, right? So for example here, if I pass a question, it will take the question, compute the corresponding embedding, then run a semantic search on it, right? Retrieve the documents which are closest to this question and use those documents as a context and then use the OpenAI LLM that we chose to answer our question based on the documents that it found to be most relevant and give us an answer. So for example, in this case I ran this query how big is stableLM? The response is the stableLM is available in 3 billion 7 billion parameter with 15 to 65 billion parameter models to follow. Right? So this is the response and then it shows the source as well where it got this information from. So in this case it's looking at the blog post from Stability AI. So it's one of the link that I provided. The next one that I asked was how good is Vicunia? Right? So in this case the response is vicunia is capable of generating detailed and well structured answers with high quality, on par with ChatGPT outperforming other models like LLAMA and Stanford's Alpaca in more than 90% of the cases. However, it has certain limitation such as not being good at task involving reasoning and mathematics. Right? So this is the response and then it also lists the source. So this is the second blog post, right? And then I asked a question regarding MPT7B model, right? And the answer was I think it was like which one is the best? Right? So the answer is the best MPT7B model is the instruct model which was trained on 1 trillion tokens. It's very subjective because there are three different MPT7B models, right? But again it lists the source. So this is how you can actually use your own website with OpenAI to create custom chatbots. I wanted to create this very detailed video to walk you guys through a step by step process both in terms of how this architecture works as well as the design choices. I will be creating a lot more content using LangChain and large language models. If you like these kind of tutorials, let me know in the comments comment section below. Okay, I think one last thing everybody would be interested is in the usage. So for this specific API key I have a limit of $5. I have been playing around with the this model for quite a while. So like in order to prepare this tutorial I had to run this multiple times and it has cost me at around 33 cents so far. So it's not really significant. However, this was just experimentation from my end. Right? If you are serving this model on your website and let's say you have a lot of traffic, the cost could be significant. Now I think it would be helpful if I can actually show you the pricing. So I don't personally have access to GPT4 API. So here are the pricing and this one is pretty expensive. So per thousand tokens you are looking at 0.$06, right? Or 6 cents per thousand tokens. But if you're using the turbo version, so chatgpt, so you're looking at $0.002 per thousand tokens. I think we were using the DaVinci model, right? So that's 0.02. I actually should have gone with here probably for some reason I thought it's less expensive anyways, if you are doing question and answer, just use the ChatGPT model. It's a lot less expensive than the DaVinci model that we were using now for computing embeddings of the your data set. So the current pricing is here. That's actually not bad at all. Right? So what I would recommend is you can potentially use embeddings from OpenAI and then you can run your own large language model such as GPT4All or even Vicuna, right to create natural language responses. So you might be able to reduce these costs. I hope this video is helpful. Now if you have a similar project, anything related to large language models and would like an expert opinion, you can actually reach out to me through my email. As always, if you have any questions, put them in the comments. I'll try my best to respond to them. If you guys want me to create content on a specific topic Also don't forget to reach out. I would love to do that. Hope you like this video. Thanks for watching. See you in the next one.