Google NEW PaLM API | FIRST LOOK

This morning when I opened my inbox, I saw this email from Google that I have access to Palm API now. So if you're not familiar with Palm, Palm is Google's new large language model. It's actually a combination of different models and now they have made it available through an API just like OpenAI. So it's going to be my very first look at this API and let's see what we can do with it. So once you click on it, the interface is pretty neat. So there are three different types of prompts. One is text prompt, data prompt, and then chat prompt. And at the bottom, you can actually create your API key, just like OpenAI. And all of this is a part of Google's Generative AI Dashboard. This is going to be just a first look at the API and its capabilities. I will be making more detailed videos on use cases and applications. And we can potentially integrate this as a part of LangChain to do information retrieval from documents. So stay tuned for those videos. Okay, so let's explore some of these options here. We're going to check these one by one. So first let's look at the text prompt. So they have a couple of sample examples. So let's say we're going to first look at a summarize a paragraph. Now in this case they have provided the prompt. The prompt is summarize this paragraph and detail some relevant context. Right. First they have provided text, then a summary of the text, and then here is another text, and we want the summary of this text. Right? So if you look at it, the preview, this is how the model is going to receive your information. Right? Here is the model that is being used. So in this case it's using text bison. Think of it as DaVinci or GPT 3.5 from OpenAI. And you can set some parameters here as well, but we're going to keep everything to default. And then you can simply run this. So if you run it now, imagine you provide your API with all this text, right? And then ask it to summarize the last section. So this is the result that you would expect. Okay. Now the great thing is they actually have this section here where you can get the corresponding code. So here's how you're going to be using this in Python. Right? So I'm going to open this in Google Colab. Just look at an example of how this will look like. When I clicked on this link opening Colab, it actually generated a whole code for it, which is pretty awesome. All right, so let's look at this, what do we need to run it? So if you have seen my previous videos, it's a Google collab way similar to, let's say something like LangChain. You can integrate it with LangChain, I think, and you just need to run the code. But we need to provide the open the API key. So in order to get the API key, I'm going to go here to get API key, then we are going to create a new one and that's going to be for this new project. I will make sure that I delete this after testing it. Okay, so we got the API key. So I'm going to copy this, go back to my Google Colab, paste it here. Right. And now let's connect to the runtime and let's start executing the code. So it's going to hopefully give. Yeah, it's initializing, so it's connected right now, in order to use it in your own projects, you'll have to first install this new package called Google Generative AI. Everything is part of that. All right, so let's run this. It's going to download the required packages. Okay. We need to actually restart the runtime. So I'm going to say restart runtime. Click yes. Okay, let's see. So this is basically running the example that we just saw. This example, right? A similar example. So it's going to be just running that, but it's giving us the whole code of how to run it. Right. We'll quickly look at the implementation and this is going to show us like how you can use this in your own applications. So it seems like we ran that, we give it the API key. Next we are telling which model to use. So by default for text generation, it's using Bison 001. They have kept the naming convention similar to OpenAI. So that is very interesting. Okay, so next are some model parameters and then the maximum output tokens that's defining here. Now the interesting thing is they converted the original text into base 64. But if you want to use it in your own applications, you need to simply provide a string here and then it will work. Right. So you don't have to do the base 64. But I don't know like why this is converted. Right. So essentially you set the default. These are the parameters for your model. Then in order to run this or make a prediction, you simply need to get palm, generate, underscore text, pass on the default parameters that you just set and then the text that you have. So let's look at the results. Okay, so when we run this, I actually got the summary and this is pretty neat. Okay, so it's a very simple way of configuring your API key and then using it. It's very similar to OpenAI. Okay, so that was just text generation example. I'm actually interested in looking at how this data prompt works. So let's create one and we will look at the examples they have provided. So first look at this opposite examples. So here they have provided some inputs and outputs. So the opposite of strong is weak, thick is thin, and so on and so forth. Now this is great because here you can actually test the model, how it's going to behave before you deploy it and use the API. Right? So as a user you can provide your inputs, some examples, and then here are the test examples as well. Let's add another one. So I'll say good and the model response should be bad. Right? Okay, now we can test it again. Let's look at the model. So it's again using the text python model. Right? And if we run it, let's see what happens. See, I'm wrong. Right? In fast and slow, Right? So as I said, this is a great feature because you can actually test it here and you will know the behavior of the responses when you test it using the API calls. Let's look at another example. So this one is what analogy Factory. So busy, like a bee. Right. Living in a bubble. Right. So in drop, let's see what it comes up with. The output drop like a song. That's pretty awesome, right? It's pretty neat. The travel agent example. So in this case what is provided, speak as if you had a travel agent sharing a general overview without bullets about location. Right. So the input is hey, things. Here's a description and here's something in excited voice. That's pretty neat. And some of the few examples. So let me add another one. Let's say Seattle, Washington. Okay. And okay, let's test it. All right, so let's see what it comes up with. Okay, here's the output. So it gave us two different outputs. I want to actually see like how this is going to look like in the API as well. So we're going to look at that in a minute. But the response for Portland and Maine is that it's a city in the state of Maine. It's the largest city in Maine and the seat of Cumberland County. Right. And here is what it would sound like an excited voice. Right. Similarly an output for Seattle as well. So this is pretty awesome. Let's look at The Python code. So it's going to be very similar to what we saw in the first example. I'm going to close out this one. Okay, let's see. So you need to provide your API key, right? Let's see if anything is different in this case. So we'll still have to provide things in the text form. Let me actually just run this and let's see what the outcome, the output looks like. Okay. I wanted to look at how the data input will look like, because in this case it's in the form of table. Right? So if you look at it here, so the actual prompt that we are providing, speak as if you are a travel agent. Right? And then we have on the new line location. So that's the Athens. Then the new line is basically the description, Right. And then I think this is the. Okay, and then there's another new line where you have the excited voice. Right? So it seems like you need to use the colon in order to differentiate it different fields or different columns, like the way you have it here. Okay. And then there's another example for another city as well. So that's how you need to provide in the form of a table. So let's look at the response, how the response looks like. Okay, so, and here is how you are going to get the response. So initially it's just the general description and then excited voice. So this is pretty neat. Pretty neat. It seems to be working pretty good. It also has a chat model. So let's look at that. In this case they have a few samples. So let's chat with an alien. Okay, so the context be an alien that lives on one of Jupiter's moon. So you need to provide the context and here's what the user will say. And then what is the response from the model? And again you can experiment here in order to test the behavior of the model. So I'm going to say how is the weather there? Right. Let's see what it comes up in the model is thinking. And then like it kind of remembers the context. So it says the weather on Europa is very cold and harsh. The average temperature is below is about minus 70 degrees Celsius. All right. And so on and so forth. So this is pretty good. Let's look at another example. So this is imagine a world. Again, you need to simply provide a context. So that will set the tone and style of the instruction for the model. I'd say here, I think this is just some example output and you can test it out here. Now the model here is chat bison Instead of text, the API looks for this or the code actually looks for this. So I'm going to open this in another colab. Okay? So don't worry about these. These are simple text messages. But I think the most important part is that you need to provide examples then the messages and the corresponding context. Okay, A couple of other things to see. If you go back to these, there is actually a prompt gallery as well. So these are some examples that they have provided. For example, let's look at this one. Let's say this. So this is who are all the people and places named in the paragraph below? Respond in JSON file. So imagine you have a bunch of information in the form of text, right? And you want to extract relevant information and put them in a JSON file, right? So this is able to easily do that because here it's extracting all the people from the text as well as the different places. Now there's a another relevant example which extract grammar from the sentences. So the input is going to be a sentence and the output is supposed to be nouns, verbs and adjectives, right? And this is a very similar table to what we saw before. So as in you're going to provide some examples and from those examples it can learn and then when you provide a new input syntax sentence, then it will be able to retrieve nouns, verbs and adjectives. So this is pretty neat for information retrieval. Okay, let's look at another one. So this is a classify a request. So basically, hey, example prompt. So you have provided some examples. For example, I would like to listen to a song by Deft Punk. So that means you are talking about music. So for example, here simple sentences and what we are talking about the kind of topic, right? And you can provide then some input examples and it will be able to predict the output. Now what are the things that you can do with it, right? So this API can actually write code for you. So here is an example that you can write code to generate multiple of a number from a given list, right? And it's able to do that. We looked at some writing examples, so you can definitely do it. Here's an editing one. So rewrite this into a casual email, right? So it will be able to get your text and write an email for you. And all of this is through the API, right? So problem solving, I guess like a recommendation system. Then you can analyze data or extract data that what we also looked at. And the last one is actually agents, okay? So you can describe different operations and then it will perform those operations. For example, this is behaving as a dungeon master, right? And here's the output based on the information that provided, like different moves to perform for the game. So this is pretty neat. Now the context window for this Palm 2 API is around 8,000 tokens, which is pretty decent and it's comparable to what GPT4 offers you. So if you haven't gotten on the waitlist yet, I'm going to put a link in the description and you can join the waitlist. Now a couple of other things. Right now it's a preview, so it's not commercially available. You cannot use this for commercial purposes. I have been playing with it for a little bit and I don't know, like what's the limit of usage? I couldn't find that information anywhere. Now they have already showcased some sample apps that people have developed on top of it, so it would be actually nice to see what people can come up with. But again, it's in a preview mode right now. Anyways it's great to see that as users we will have a lot more options now. So it's definitely a direct competition to OpenAI and the models that Google is offering, they are very powerful models. So I will be exploring this API further in my upcoming videos. So definitely keep an eye out for those videos. They have provided very comprehensive documentations to get you started. Anyways, if you want to keep yourself up to date with the latest advancement in machine learning and generative AI, check out our Discord server as well. And don't forget to like and subscribe to the channel. As always. Thanks for watching. See you in the next one.