How to use Google PaLM API for Free

Did you know you can use OpenAI's Palm 2 API for absolutely free? It's a worthy alternative to OpenAI. You can build powerful applications without breaking the bank, at least for now. The best part is it comes with its own embedding model. That means that you can develop information retrieval systems using Palm API. In this video, I'll first show you how to access and use this in Python. In the second part, we will look at how you can build talk to your Document app and integrate this within LangChain. Palm API is currently available as a part of the Google's Maker suite. I think it's not generally available, but that could change at any moment. So you will need to sign up for an account to get the access. I'll put link in the description. You can go to the Palm API page and then click on get your API key. Okay, so this is the interface of the Google Maker suite. I have covered this in my previous video, but right now we're just interested in getting the API key. So let's click on this. This will take you to another interface where you can create a new API key if you want. But I have an existing API key, so I'm going to simply copy this and I will delete this after recording this video. Okay, so in the first part of this video I'll show you how you can use the Palm API in Python. And in the second part we will build a fully functional app that will let you talk to your documents. Now, in order to use the Palm API key, you first need to install the Google Generative AI toolkit. So you can do that using pip. And after that you are going to import PAM from the Generative AI too good. Okay, so there are a number of ways in which you can set up the API key, but here is a very simple process. So you call this configure function pass on your API key and that will configure the API in your environment. Now, there are a number of models that are available through Palm too, but for text generation there is currently only one model available through the API and that's called a text BISON001. So here in this code we are passing through all the models that are available to Palm API and then we're specifically looking for the text generation models. Now the question is, how exactly do you use this in your own programs? The first I created a new variable that is storing the name of the model and then we need a prompt. Now here I'm going to be showing a very simple example of text Summarization. So in this case, the prompt is summarize this paragraph in detail, some relevant context, and we are doing a few short promptings. So here I provided an example text and then the corresponding summary, and then we are providing the text and we want the model to generate the summary for this text. Okay, so how do you use the API? Here is how the syntax looks like. So you have the Palm object, then you call the generate text function on it, pass on the model, assign the prompt that you want to use, then pass on the temperature, and then the last one is maximum output tokens. So I'm setting it to a smaller value. Now, when you run this, you actually get get a response pretty quickly. Now, the summary that it generated is actually really good. But in this video we are not really concerned about the performance. I just want to show you how to use the API. Now, if you're interested in learning more about the capability of Palm API, I have a detailed video on that and I would recommend you to watch that. Now let's look at how we can integrate the Palm API within LangChain. Okay, for the second application, we're going to be looking at this simple app that I have put together. So this is basically a streamlit app that uses LangChain for information retrieval from PDF files. In one of my previous videos, I showed you the code of how to build this using OpenAI embedding model and the LLM by using the OpenAI API. But in this video, we're going to replace everything with the Google Palm API. So here's the code. If you have been following my channel, you probably have seen this part of the code. So essentially we're importing streamlit, then loading this ENV for loading the API key, Then using recursive character text splitter, there's a vector db. I'll go into the detail of this. For document loading, we're using the unstructured PDF loader and there is a retrieval Q and a chain. Now, the only difference that you will see is this. So here we are using the Google Palm embeddings for creating the vector store embedding. And for LLM, we're going to be using the Google Palm LLM. Okay, the next part of the code is simply creating the sidebar with some of the information. And then we have this load.env function. So essentially what we are doing is that we are loading the API key from a dot ENV file. I will delete the API key right after creating this video. Now, for this example application, I'm using the original Orca paper as my source document. Here we are putting everything together in a main function and you probably have seen some of these steps. If you haven't seen these, I will recommend you to watch a couple of my videos. Links are going to be in the description. So the first one is basically we're defining a path to the PDF file and then we're using the unstructured PDF loader to load that specific file. Now, you can have multiple files, you can give the paths here, or you can directly read multiple files from a source directory. Next we are creating the vector store. So there are multiple components that we need to consider. First is this new embedding model. We're using the Google Palm embedding model. After that, once we load the documents, we actually want to divide it into different chunks, right? So for that we are using the recursive character text splitter. Chunk size that I'm using is 800, but you can play around with it and an overlap of 0. So no overlap in this case. Right. So essentially what is happening is we load the file using the loader, then divide it into different chunks, and then use the embedding model to calculate or compute embeddings for those chunks. Okay. And put them as a vector store. And now by default this is going to be using the Chroma DB vector store. Next we need an LLM. So here I'm using the Google Palm LLM and this is basically a replacement of the OpenAI LLM that I had before. After that I'm creating the retrieval Q and a chain, right? So we need to pass the LLM the type of chain that we want to use. So here I'm using stuff, but you could do MapReduce or Map re rank, there are different options available, Right? Next we pass on a retriever. So that's basically the index that we created. And I'm also enabling for this to return the source document. So basically whenever we ask a question, it will also show us what chunks it used in order to generate that answer. So next we are simply creating text input field that will get input from the user. Now, whenever there is a new input from the user, we will run the chain, get a response. Now, since we have enabled the source documents, so the response is going to have two parts. One is going to be the result that we get. So that's basically the response generated by the LLM. And the second part is the source documents that the embedding model is returning. And we simply wrap everything within this main function. Now, in order to run this, we will need a whole bunch of different packages. So, so I have put together everything that you need within this requirement Txt file. So the rest is bas standard stuff. We first need to create a virtual environment. We're going to be using conda for that. And I'm going to call this Palm, because I have already created this version environment and we're going to be using Python 3.10.0, right? So it will ask me that this already exists. Do I want to remove it? So I'm going to type no, but in your case you need to type yes. All right, so let's clear this up again. Next, I'm going to activate my virtual environment by typing conda activate pub and you see we have a virtual environment activated. Now, in order to run this, we're going to be using streamlit. So the command is going to be streamlit, run and then name of the file that we were working with. So this is called Palm Chat PDF. So I'll hit enter. Okay, so we have the app running on localhost at port 8501. Now, keep in mind that it takes quite a bit of time to do the embedding computation and to load this user input box. All right, so once you have this, you can start asking questions. Now, okay, so we will start with a very simple question. Who are the authors of this paper? Okay, so we'll start with a very simple question. Who are the authors of this paper and what are their contributions? Okay, so we get the answer. So here's the list of authors of the paper. It seems like it got the list wrong. So the first author is actually right, but for the rest of them, they do not exist in the paper. Now, if you look at the retrieved chunks, this is basically the contribution section, I think, right? It says author contribution listed at the end of the paper. So here's the corresponding author. And then I don't think these are the proper chunks that we would expect it to be retrieving. Now, somehow it picked these names from the references section. So I'm not sure what exactly is going on in here. Okay, so next I asked the concept of instruction tuning. So it came up with a pretty good answer. But if you look at the chunks that it has returned, so for some reason it's simply returning the same paragraph multiple times. It could be that the similarity metric is not really a good metric in this case. Now, this might be an ideal situation to use something like maximum marginal relevance instead of similarity score. Okay, so the goal of this video was to show you how to use Form API using Python and then how to integrate it as a part of LangChain. Now, for the time being, Palm API is free to use for certain countries and it seems to have some sort of rate limit, so I think there is a limit of 90 queries per minute. So check it out if you get access to it and let me know how accurate is this compared to something like OpenAI? I will be creating more videos where we're going to be doing a direct comparison between OpenAI API and the Palm API. If you want to see the original video in which I presented this app, I'll put a link to that video. We are using OpenAI embedding as well as the LLM. Hope you found this video useful. Thanks for watching and see you in the next one.