Getting Started with Gemini Pro API on Google AI Studio

Google just opened API access to their Gemini Pro models to the public and the best part is you can test this for absolutely free. We're going to look at the pricing in a little bit, but Gemini Pro is the second best model from Google and it's a multimodal model. In this video I'll show you how to use both the Vision as well as the text version of Gemini Pro. Through their Python SDK, Gemini Pro already has integrations with tools like LangChain and Llama Index. That means that you can build rag pipelines on top of Gemini Pro. We will cover that in a later video. Before showing you how to use this in your own projects, let's talk about the API pricing on their pricing page. They seem priced to help you bring your app to the world. If you are making less than 60 queries per minute, it's absolutely free for everyone to use at the moment both in terms of the input as well as the output. The only catch is that Google will use this data, both the input data that you provide as well as the output from the model to improve their products. If you need more than 60 queries per minute, you can opt into pay as well. That is not yet available, but I think it's going to be available pretty soon. In terms of the price both for the input as well as output tokens, it's actually pretty good compared to something like GPT 3.5. So here is the price for GPT 3.5 Turbo and if you compare, the price of Gemini Pro is actually an order of magnitude Lower than GPT 3.5 Turbo and you also have the ability to process images. Again, if you compare the Gemini Pro vision model with GPT4 Vision Preview model, the price for image completion is also lower. Now, just like OpenAI, if you pay for Gemini Pro API usage, then Google is not going to use both your input as well as output data to train or improve their products. So again, the best part is it's absolutely free. If you are just getting started now, let me show you how to use this. Gemini Pro is currently available within the Google AI Studio which used to be called Maker Suite. Within the Google AI Studio you can test the models. Currently there are two different models available. One is the Gemini Pro which is the text model. The second one is Gemini Pro Vision which has the ability to understand images. You can experiment with both of these models in here. It's just like the OpenAI playground. However, if you want to use these models within your own applications, then you will need to create an API key and use that in your own code base. Before looking at that, let's just experiment with the models here and let me show you a few very interesting options that Google has added. Okay, so I'm going to use this test prompt, what is the meaning of life? Just to look at the output. Now, in this video, we are not looking at comparing the output from Gemini Pro to something like GPT 3.5 or GPT 4. I'm going to create a subsequent video on that. The goal of this video is just to show you how to use Gemini Pro in your own projects using the Python SDK. Okay, so you can see we got an output from the model. So let's look at some of the options that you have in here. So you have, you can set the temperature. Currently it's set to 0.9. One very interesting thing that Google has done in here is that it is giving the users the ability to define the safety settings for the model. They have four different harmful categories, Harassment, hate speech, sexually explicit content, and dangerous content. And you have this slider that you can use to actually set different levels, which is very interesting. And it gives the user, the developers more control on what they want their users to be able to see. So I think it's a really good initiative from Google. Now you can also set some other settings in here. So for example, this is the maximum number of output tokens, top k, top P. Now, once you're happy with the model behavior, then you can simply export the code with all the settings. So just click on this get code. This will give you the Python code in this case, but you can also get the JavaScript and here's everything that you need. So these are the configuration settings that we just used. Here are the safety settings. You can modify them and then how to actually use the model itself. We are going to look at an example later in the video. If you click on the Gemini Pro Vision model now, you will have the ability to upload images as well. If you want to use this in your own projects, you will need to create an API key. So we're going to click on this Get API key. And here I already have an API key that I was testing, but you can create a new API key for your project. So simply click on that. Just copy your API key. Now let me show you how to test Gemini Pro in a Google Colab. So in this Google collab we are going to be looking at a few things. The first one is going to be how to set up your development environment and how to Set access to your API key within Google Notebook. Second, we are going to look at how to generate text responses from the model, then how to do streaming of those responses, as well as how to use the chat model. Later, I'll show you how to use the embedding model that you can use in your own Rack pipelines. And I'll also show you how you can interact with images using the Vision version of Gemini Pro. We need to set our API key. So click on this key option. Now here you can add a new secret. So I currently have one which I'm calling Gemini and I provided my API key in here if you want to add another one. So for example, let's call this test. Then I'll provide the API key in here and let's just enable it so that it's visible to your Google Colab notebook. Now, in this case you need to remember the name that you assign in here. That is going to be your environment variable. Once we do that, the first thing we need to do is to download and install the Google Generative AI package. Here we are just importing all the packages that we are going to be using. Now in this case we are using the the user data function or object from the Google Colab just to retrieve the API key. If you're running this locally on your own machine, you can set an environment variable and retrieve it that way. And at the end we are defining a function just to show the responses generated by the model in markdown. Next we need to retrieve the API key. So if you recall, I had this environment variable called Gemini. So here I'm just providing that. And we are going to set this in configuration. Now if you are running this locally, you can set an environment variable called Google API key and then load that in order to use it in your own code. Next we are going to look at all the models that are currently available within the Google Generative AI package. So currently we have access to only the Gemini Pro, which is the text model, and the Gemini Pro Vision, which has the ability to understand images. And as I said in the beginning of the video, there is a rate limit of 60 requests per minute or 60 queries per minute, but it's absolutely free to use, at least for the time being. Now, how do you actually use the model? So we are going to be calling this Generative model function. On the Gen AI object that we created, we pass on the name of the model. So in this case we want to use the Gemini Pro version which is the text generation model, and that will Load the model for us. Now, in order to generate a response from the model, we will need to call this generate underscore content function on the model and pass on our prompt. If we look at the response object that we got, there are a lot of things that we can call, but the one that we are interested right now is just the text part of it. Let's run this. This is basically the text or response from the model that was returned. And using the markdown function that we wrote, we can convert the this into a nicely formatted markdown. So here is the response that you see in markdown. Just to repeat what we did so far. So initially we imported the Google generative AI package as gen AI. Then we said that we want to use the Gemini Pro model using the generative model function. And after that we called the generate content function and pass on our prompt and we get a response as a text field. In terms of the API implementation, it's a very clean implementation and I really like how it's formatted. Now, apart from the text, there are some other properties of the response object that we want to look at. One of the most important one is the prompt feedback. So basically when the model generates responses for your prompt, it looks at the prompt and assigns it probability based on the four different harmful categories that we defined. So for example, if you look at in this case my prompt was what is the meaning of life? And then it looked at the safety ratings. So for example, if the category was sexually explicit, the probability of this category being present is negligible. And same is the case for hate speech, harassment as well as for dangerous content. Later in the video I'll show you how you can control this for different prompts and allow some of these things based on your own tolerance. If you have used banned, you are probably aware that Bard generates multiple drafts and show you one of them. Google has enabled exactly the same behavior to their API as well. So in this case on the response object there is another property called candidates which will show you different candidates or different responses that it generated and you can select the response you want out of it. Currently it's just limited to generation of a single candidate, but it seems like they're going to expose multiple responses to the user and then as a developer you can choose which response to show to the user. So for example, you can set some of the configurations here. So apart from the simple prompt that you get from the user, you can set a few configurations. Right now the candidate count can only be set to one, but in a Future update, this might change. You can control the maximum number of output tokens. So far we did the whole text generation at once. But sometimes you want to stream the text. That means you want to generate text in chunks and show them to the user. So in order to do that, all you need to do is just set this stream parameter to true. Now, once you run this, you will get a response, but you will need to retrieve chunks from the response and show them to the user one at a time. So, for example, here's the first chunk of text, then the next, and then the next, and so on and so forth. So far we just looked at an example of using Gemini Pro as a text generation model. However, you can use this as a chat model as well. The way you do it is that you create a model. So specifically, we're using the Gemini Pro model. Then instead of content generator, you want to use this in chat mode. So for that, you're going to call this Start chat function and you will pass on the history. Now, in this case, we are passing on an empty list, but you can pass on previous conversations that you had and that will become history to the model. Now, in order to use the model, so you are going to call the send message function. Here's an example prompt in a single sentence. Explain how a computer works to a young child. You get the response and we can show the response in here. So the response is a computer is a machine that helps us do many things by following instructions we give it. Now we can actually look at the history. So everything is divided into parts. The first one is text input from the user, and that's why you see the role user. Then we have a second part which is the response from the model, and the role is set to the model. Now, you can store this history as a list and you can provide this to the model when you initiate it. So it's going to use that in its chat history. Or you can simply continue the conversation. So you can again call the send message function, ask another question, or pass on another prompt. You can also stream the responses if you want, and you will get a streaming response. In this case, we are retrieving the role as well as the corresponding text messages, right? So you have the user input, then the model response, another user input, and another response from the model. As you can see, that since it's a chat model, so it keeps all the historical conversation that has happened before in order to generate more responses before. Looking at the Gemini Vision model, as well as how to change the safety settings let's look at the embedding model that Google has released as a part of their Generative AI package. This is a purely text embedding model that Google released and you can use this for a number of applications including anomaly detection in your documents, clustering with embeddings, as well as Document Question answer as a part of the RAG pipelines. I'm going to be creating more videos on this, but let me show you how to use the embedding model. Something that Google has done in here is there are five different tasks for which you can use the embedding model to compute the embeddings and it seems like these are task specific embeddings, which makes it very powerful. Within the generative AI package there is a special embedding model you can invoke that using the embed content function. Currently there is only one model. So embedding 001 then you need to provide the text that you want to encode, then the type of task that you want to encode it for, right? And if you are doing a retrieval document, then you need to also provide a title for the embedding that you create. Now there are five different tasks. Retrieval, query, retrieval, document, then semantic similarity, classification, as well as clustering. The embedding vector that you get has 768 dimensions, so it's a pretty large embedding vector. Instead of providing a single sentence, you can provide multiple sentences. So for example, if you look here, we have three different sentences and we get three different embedding vectors for each of this sentence. You can also provide whole paragraphs and this will give you embeddings of the paragraph. Both Llama Index as well as well as LangChain has already support for this embedding model. So in a future video I'll show you how to use this embedding model as a part of your RAG pipeline. Just like the Google AI studio, you can control the safety settings within the Python SDK. For example, here I defined or initiated a new model. Then I asked it how to break into a car and the response is I'm sorry, I'm not able to provide assistance with illegal activities. Breaking into a car is a crime and I would not be able to help you with that when I looked at the safety ratings. So for some reason it identified the prompt to have a low probability of containing harassment, although I was expecting it to have a high probability of dangerous content. So let me show you how you can potentially change this behavior. Although personally I did not had much luck. So you can define your own safety settings. So again, you have four categories. Harassment, hate speech, sexually explicit content, and dangerous content. And then you can define different thresholds. So for example, for the first three, I defined block medium and above. And for the dangerous content, I said block none. But even after that, when I ran the same prompt, so I got this response, it might be that it's detecting some illegal activities, and as a user you cannot really change those in here. So that might be a possible reason that it's not working for my prompt. Where are these settings from? You can actually look at the documentation in here and they have an explanation. For example, block none. So you set it to block none. Then there is block few, block sum, and block most. So you can set these based on different thresholds that you have in here. I'll put a link to the documentation. In the last part of this video, I'll show you how to work with the Gemini Pro Vision model. But for that, first we need an image. So here is an image. This is provided an example notebook from Google. We downloaded the image. Then using the pillow package, we are reading the image. And this is an image of food. Now, since we want to use the vision model, so we are going to initiate another model. And this time we're going to be using the Gemini Pro Vision. So now the model is going to have vision capabilities. And if you pass the image as an input, the model will generate a response. And this is basically what it thinks about the image. So in this case it says a chicken teriyaki meal prep bowl with brown rice and roasted vegetables, which seems to be accurate. The beauty of this vision model is that not only you can provide images and input, but you can also provide text. So for example, here we have a text prompt along with the image. So here's our input image. And then the text prompt is write a short, engaging blog post based on this picture. It should include a description of the meal in the photo and talk about my journey. Meal prepping. Based on the input image, as well as the input prompt, it generated this response. Meal prepping is a great way to save time and money and it can also help you eat healthier. Based on the image, it's able to identify that there is brown rice, roasted vegetables and chicken teriyaki. So it's able to include that information in the response. This is pretty awesome. As you can imagine, this opens up so many possibilities, even with the chat with your documents or RAC pipelines. Now you can use this model as a part of a multimodal RAC pipeline, which is going to be pretty awesome. In a subsequent video I'll show you how to do that. I'll highly recommend everybody to check out the documentation that Google has provided both for the API as well as there is a prompt gallery. So Google provided a few examples of how to interact with these LLMs and also the vision models. I hope you found this video useful. Consider liking and subscribing to the channel, and let me know in the comment section below if there are specific topics related to the Gemini Pro API that you want me to cover. Thanks for watching and as always, see you in the next one.