Using Large Language Models with the Cohere API

Foreign.

And thank you for joining today's Code Along. My name is Rhys and I'll be your moderator today. We're going to kick off today's session in a couple of minutes. We're just waiting so everyone has the chance to join. In the meanwhile, though, we'd love to hear from you. So let us know where you're joining from using the chat or the comments, depending on what platform you're watching from. And yeah, tell us something that you'd like to get out of today's webinar. Today's code along is hosted on datacamp Workspace. I'll be posting a link shortly in the chat so that you can code along with us. So yeah, if you don't already have a Datacount workspace account, please sign up for one. And yeah, you'll be able to code along with us as well. Also today, if you'd like the recording and any of the resources sent to you afterwards, if you need to leave early or anything like that, then please register for the event. You can do that either by heading over to our LinkedIn page, or I'll be posting a link to do so in the chat shortly as well, so keep your eyes built for that. Brilliant. I think that's everything from me for the moment. I will be back to repeat these messages for any new joiner shortly, but until then, enjoy the background music.

Sam.

Hello everyone and thank you for joining today's Code along. My name is Rhys and I'm going to be your moderator today. We're going to kick off today's session in about a minute or so. We're just waiting so everyone has a chance to join. In the meanwhile, though, we'd love to hear from you. So let us know where you're joining from using the chat or the comments, depending on on what platform you're watching on. And yeah, tell us something that you'd like to get out of today's webinar. We are doing the code along today, so if you want to code along with us, make sure you've got a datacamp Workspace account. And yeah, keep your eye out for the link that I've sent in the chat. I'll be sending it again shortly. But yeah, keep your eyes in the chat for the link to code along with us, as well as the link to register for the event so that you get the recording and the resources sent to you as well. Brilliant. I think that's everything from me. So now I'll hand you over to your host for today's session. Richie. Richie, please take it away.

Hi there, Data Scamps and Data Champs. This is Richie. Now we've had a few code alongs recently where we've explored using large language models for, well, a variety of tasks. And I'm very pleased that today we got a world premiere for you all. This is our first ever session on the Cohere API.

So.

So I hope you're all dressed up fancy for this premiere. And we've got a really fun use case around product descriptions which is also incredibly useful. So I'm sure whenever you've done some online shopping, you'll know that product descriptions are often absolutely rubbish. And so getting AI to help make them better is a huge win for shoppers all over the world, not to mention the people who have to write those product descriptions in the first place. Now our guest today is Rishit Dholakir. He's a machine learning engineer at Cohere where his job is to help build and improve the company's large language models. And he's got a wealth of experience in both natural language processing and deep learning across banking, retail and ed tech. So with that, over to you Rishit.

Thank you Rishi for the introduction. Hello everyone, who's joining in? So I'm Rishik, I'm working at Cohere, as Richie said, as a machine learning engineer. And today we just starting off with some introduction. That's what we are going to do. What is our goal and even as to why we should even pursue learning about Cohere or any sort of natural language or machine learning API. So let's get started. So about this case study today that we'll be talking about. So with respect to data that we're going to use, we're going to do two parts of it. One is we are going to generate data using the Cohere API, such as product descriptions, product titles, and we are going to use a product data set which is something that most of the companies or people might have it in their databases and they would like to make maximum use of those and use artificial intelligence in order to extract information from them. So that's the way where we are going to use data. Then we have a primary goal which is for us to understand how we use Cohere API so you all can build your products and power them with AI. And the secondary goal is to get an insights into like what are the capabilities that LLMs have, which is not just generating text or something, but there's something more and creative that you all can do it using just the APIs, rather than thinking and scratching about how do I maybe train my own machine learning models or maybe even think about how do I get this infrastructure. So Cohere provides you this API where you don't need to worry about all that and just focus on your software development process and make creative and amazing products. And the last question is like, why do you attend a webinar or related to AI or related to Cohere? The thing is, because in this generation now, AI has become the new fuel, I would say, and with the help of AI, you can create so many intelligent agents that will not just help ease your life personally, but even help you immensely improve your productivity. So that's from me about the case study. Now I'll talk a bit about what Cohere provides. So Cohere provides you foundational language models. So these are huge machine learning AI models that are trained on lots of text. And in order for a person to normally train all these sort of texts and everything else, it takes lots of millions of dollars of investment and lots of other soft infrastructures and hiring machine learning experts and all which might be a tedious task for any sort of a team. So Cohere provides two sorts of models. One is the generative model and the embedding models. These are the two ones that I'm going to discuss today. And with the generative models you could do multiple sorts of tasks like text generation, text summarization. And with embedding models you could do classification search. That's something that we'll be looking upon. We even have a latest offering that just came out which is the Retrieval Augmented Generation, which I'll just talk a bit about at the end of this session. Next we'll go on to some of the different LLMs that Cohere provides via its API. We have something called as the Command and the Command idly, which are the flagship models of Cohere for generation tasks. And the command model is something that it's a larger model which people can use. The Command light model is something that people can use for making chat box, which has low latency and which works much faster. We have the embed models, which are not just in English, but we have a multilingual model as well, which actually can work across multiple languages and can be used to embed most of your text for doing classification semantic search. So this was just a bit about the different models that Cohere has to provide and the API end of it. Now we'll go on to our workspace. I hope everyone can just open up their workspace. Yeah, so cool. Let's get started with it. So welcome again to the code along webinar. This is just a description for people who would want to even look into it later on to see what Cohere does and what are the different APIs at word. You all could just click on this link, for example, and you should be able to go to the website to see and read more about the APIs. Yeah, first we all start off with sign up. So you click over here into the sign up, then go on to try now on on the website. Or you could just skip that and just go straight away to sign up and then enter your email address. You should be able to create an account and once you create an account, you could be. You should be able to log in and go to the API key tab. Once you go into the API key tab, you all will have your free trial keys that we'll be using for this workspace and for this code along today. I'll just give it a minute so that people can, you know, create an account and be able to, you know, access the API keys. Okay. Yeah. So now I'll start off with setting up the workspace integration, which means that you have your API key, it's personal to you, you don't want anyone to misuse it so that, you know, you get build up for all the amount of the tokens that have been used or the tokens that have been generated, the requests that have been sent across to here. So you all could go to Environments tab over here onto your left. And once you go into the Environments tab, you can click onto the environment variables in your workspace. You could enter the name which is Cohere underscore key, that is the name of the key. And you could paste in your API key inside the values over here and press Create. Yeah, you all can hopefully follow those instructions. And now once that is done, we'll have to do some installation of packages, for example, Cohere itself because it's not pre installed on the workspace. So we'll start off with that initially. So let's copy this workspace and let's click. It does take a while for it to install all the requirements that are there. Now once you're done with installing the requirements, please do make sure to restart your kernel because some of the dependencies need to be activated once the kernel has been restarted. Yeah, now once your kernel has restarted, all you need to do is just copy out these imports and add your Cohere key. Into this particular variable that we have in order to activate it. So let's do that. Yep. I am giving about another 30 seconds or so so that people can just catch up a bit in case. Yeah. Okay, so we're just done with all the housekeeping work now. Let's begin. So consider yourself to be. This is sort of a use case where I've just come up with. So let's consider that you are a tech founder of say, a company called Nano Goods. It's an E commerce website where you sell wholesale Items and do B2B businesses for many other people and you help optimize some supply chain and all the other sort of things with respect to establishing and accomplishing B2B businesses. And now what's happened is that it's a new company, there are very few people in and you're actually short of employees who are in the creative space who are able to make good content for you. For example, in sense write descriptions for your products that you're able to import from somewhere or you know, even make a product title for you. So for that reason what we'll do is we are going to use the Coheres Code Generate API to create some interesting, I could say product titles and product descriptions. So for us, what we'll be doing is we'll be using the command model. So Coherent provides three sorts of model. One is the command model, the Command Light and the Command Nightly. The Command nightly is like an experimental model that gets updated every week so you don't have to wait for new new releases. And over here, when you look below, we'll be looking at the code Generate API. So the co Generate API consists of a few things. One is entering your model name, which is which model are you using? We'll be using Command Nightly today. Then it's the prompt. The prompt. The prompt is mainly you could say a text which contains the instruction or the idea that you want the AI assistant to help you out with. So you could just add sort of a text which will be looking further as to what sort of text we're going to write. Then we have something called as a max token. So you want to say you don't want to generate like three or four paragraphs of a description. You want to generate short and concise descriptions. So you could even say what's the number of maximum number of tokens. You should let the model generate. The other thing is the temperature. The temperature is mainly a tool which could be used for making sure how creative and interesting your content is when the Generations happen. So it's to, it's a control that's given to you to determine how interesting and, or creative do you want your content to be. So you could say a bit, you could have temperature between, like, between like say 0 to 5. We'll be considering a bit of a lower temperature today just so that it makes more sense because you go towards really high temperatures, you might end up, you might derail from the basic idea of what your prompt or what you want the AI assistant to do as such. Next we say, is the stop sequences. The stop sequences is technically a list of sequences in order for a model to identify as to when do you stop a particular generation. So you could say that the first time I see a period, I would like to stop the generation. Or the first time I see a new line, I would like to stop the generation. And then you have something called as a return or likelihoods. So the likelihoods, technically you could say is how likely a word is to be present in this next sort of a generation for a particular prompt, which means that what is the probability of selecting that particular word now that we have the particular word being selected? You could even return the likelihood or the probability of these particular words that are there. So let's start with our exercise one where now we are going to ask any assistant that since we don't have any creators or merchandise specialists who can help us with, you know, creating product titles or product descriptions for your product, we would say that we would, we would actually instruct the model saying that, okay, you are an Apple merchandising specialist. Now you need to draft a product title and a product description, say for a smartwatch from Fitbit, and we give it more description as to what so that the model understands what sort of generation are we trying to provide and whatever description we want. So we say that it's a fitness brand, it's in the new ocean blue color and has accurate tracking. Now, in order to execute this, let's first copy this particular code snippet over here. And now we have this prompt which is just a normal string text. Let's just remove the string text and enter our variable text that we have and say we want 200 tokens at the max to be generated. Yeah, I'll just execute this one cell. Yeah, sorry. This was an execution that had happened before. So yeah. So here we have a product title that has been generated by Cohere's code or generate API. So it says Fitbit Versa 3 health and fitness smartwatch and it says the color as well, Ocean Blue. And now over here you have a native description that has been generated as you all can see. So it says like you have get ready to enhance your healthy lifestyle and keep track of your fitness goals. Because we said that we had a fitness brand and we say that it's offered in our new Ocean Blue colorway. As you can see, it even picks up the new Ocean Blue Colorway. This way we are able to use co generate simply by just inputting our text and generating these titles and descriptions. The other pro tip I would like to give over here is that the way you're able to instruct it more, or you could say command or provide more specific instructions to the AI model, the better it's going to be at generating the sort of text that you would want and be more creative with it. Yeah. Any questions, Richie? If anyone has, I'll just take a small break so people can catch up to you.

All right. We don't have many questions from the audience yet, so if anyone in the audience does have any questions, then please do add them to the chat. Now, I think for now we can just carry on with the code along.

Okay, sure. Yeah. So now this way you've been able to, you know, generate product title and product descriptions and you're able to make a catalog of your own. Now we'll come on our second exercise which is something called as an alternate product search. Now say for example, your company is running really well. You've got many clients that are buying your products and that to enlarge quantities. And now you have an inventory management issue that happens on your tech side where say inventory managers facing issues with out of stock products. So say a client has a product that wants a new stock of say some sort of toys that has gone completely out of stock that is not present in your inventory or say from the manufacturer side. And then the client asks you that, you know what, provide me an alternate product, not the same one, but provide me an alternate product. And they might say that they might have their own sort of restrictions in the sense that I want the same sort of a product but with a similar dimension or I want a similar product with the same pricing or I want a similar product, say, which can help me increase my profit margins. Now what happens in this case is that you don't have a catalog or you don't have annotators who have actually made it database containing all these sorts of attributes where you have a price range where you have a price range that connects with your product description and your product Name a product title. So over here we are going to use an interesting case where we are going to build a semantic search engine. And the semantic search engine would do is it would take in initially your query, which is your document, which is your item that is out of stock. And it would generate and search, sorry, not generate, but I would say it would search for other similar products. So a question would be like, how is semantic search different from say a normal search? Now a normal keyword search would be you just match up keywords in your context and you would provide search results based on it. But what happens in case of semantic search is the search is done based on the meaning of the sentence and based on the meaning of a sentence, you even if the sentences don't have similar looking, if they don't have the same words that match, but they have similar sort of words through synonyms that they understand, they'll be able to, you know, retrieve those sorts of documents, similar documents that you require for your query. So over here what we'll be doing is two parts to it, where first we'll be building a semantic search engine where we're able to retrieve similar products for the product that is out of stock using our CO embed API. And then what we are going to do is something called as dimension extraction. Because we say we have a client that says that we want a similar product but we want almost similar dimensions. We don't care about the price point for that. Say we don't have another database that consists of a category with respect to a product name and a dimension from the context on from the product name. We'll be extracting the dimension as well. For that we'll be using code generate. Once we do that, we will rank them on basis of the closest dimensions to the reference product that has been out of stock. So this is the pipeline that we'll be following where we have the product name that goes into a semantic search engine to find out the alternate products. Once we get the alternate products, we extract the dimension parameter from these product names and then use them to rank all our search recommendations for the product that has been out of stock. Hopefully that's clear. Richie, any questions? Does anyone have maybe I just so that they're clear about this point.

Actually we have got a couple of questions from the audience. Oh, sorry, one question. Let's put that on screen. So this comes from Alex. Alex says can cohere crawl a website rather than providing a CSV file or do you need another tool to do do this?

So this section. Yeah oh, yeah. So from the previous section, when you use Coda Generate, it doesn't crawl any sort of website or CSV Right now, this is all embedded within the model that has learned across multiple data sets. And all of that we have trained at Cohere. So you don't have to worry about it looking into another sort of search a website where it does all these other generations. All these generations are some things that it has learned over time, over multiple sorts of training that has been done on the model. You wouldn't need to worry about the fact that it's going on any search engine or anything like that.

Okay, so this is all stuff that's built into the large language model rather than.

Yes, yeah. Yes, yes. Okay.

All right.

That was the only question for now, so you can continue.

Yeah. So in order for us to build this semantic search engine, we'll be using CO embed code. Embed is an endpoint or an API endpoint that Cohere provides for embedding your text and your queries and your documents. And the reason why we do that is it converts these embed in. It converts these texts into meaningful representations which can be later used for classifications, or it could be used for semantic search or toxicity detection and all these other sorts of methods. And this is all provided just to an API. You don't need to train any sort of a model at the moment. You can straightaway just use it out of the box. And co embed is multilingual, which means there is a multilingual offering as well to code embed, which means that you can do multilingual semantic search, which means you can add a query in a, in a different language and get results as well in the different language that you want. So that's like a multilingual semantic search. And you could even do something called as a cross lingual search, which means that you ask it a query in one language and it will return a result to you in another language if that's present for you within the sort of documents or the data set that you're providing for it to search on or for it to do some sort of classification. So that's a really powerful tool, I would say, because a lot of diversity and reaching out in terms of a huge number of customers and audiences. So this is semantic search, where you all could read a bit about what semantic search is still more. And to be honest, semantic search can be used to even power your internal web search engine, where you could write a Python script or any sort of a script in order to, you know, say, write a natural language, to extract a document of a particular date or say a document that contains some sort of a task that you wanted to do. So it's a really powerful, it's like a sort of a search engine. Just build by yourself for your own requirements. And the other powerful use case would be like creating a stack overflow assistant, say for your documents using natural language queries. Because we know that even as coders, a lot of time takes a debug. You need to search in a query on say, Google Search or some other browser. And we have to go through a lot of. You have to go through multiple sorts of documents to understand what sort of where you're trying to debug. So this is a powerful tool that way. So over here we're going to follow three steps. One is, we will proceed with the following steps is get the product names that we have for our data set that we need to build and which becomes like an archive for us. And the other one is embed these archives. It means all these product names need to be embedded. And since we need to be able to give a representation for all these, and then we do a search, nearest neighbor search on the query, which is the item that we provide as the one that has been out of stock. Yep. So over here we have this one particular imports that we need to do. For example, we need to import Numpy, we need to import Pandas, we need to import Torch, and we need to import Regex. So re. So let's go ahead and execute it. I'm supposing many of you might have to import these. I'm just leaving it on the screen for another 30 to 40 seconds in case people need to import these packages. Yeah, Once you all are done importing these, we are gonna use a data set, an Amazon data set that has the product names. And we are going to retrieve just the product names for our use case today. Yeah, we'll execute that cell as well. Now that we have these product names, it's in a sort of archive, it's in a sort of represent. It's in sort of like you could say a database. And now we need to pass through our representation language model, our embedding model, and so that you could end up getting like a representation archive of it, which means a representation of each of these particular product names. Instead of text, you have numerical representations of these. So we'll be using code embed and quite a few things from pack here. So the code of embed uses text and the Text over here is your product name and the set of documents that you want to, you know, embed. It's a list of strings that you can provide and then you have the model. So the model over here is your embed English Lite version 2. That is the Cohere embed model that we'll be using. So there are other models that Cohere provides, which is the English large model, English small model and the multiple model as well. Yeah. So we'll be using this model which has a representation of I think 4096 dimensions. I think that's the one we'll be using. We have another argument called as truncate. So the reason why we use truncate is that the embedding models were required max context length or token length of 512. So any context, any token length in the string which is greater than 512 will be cut out and you, you'd be able to truncate them to just 5 and 2 number of tokens that would then be passed on to get its representations. And there are two ways you could truncate them. One is you could truncate them at the end of the string that you have, or you could truncate them using the start of the string. So that way what we have done is we'll be able to truncate these particular strings if they are have, if they have a context more than five and two. Let's execute that over here. Yep. Once we are done with that, what we'll be doing is we're going to do three things. One is we'll convert this into a numpy array, we'll check its shape and we will convert these embeddings into a tensor using a TORCH tensor. And the reason we do that is because we need it to, I could say use it for our similarity check. That we do later on after we get all these embeddings. So we'd use, I think I did use np. Yeah. So we'll use NP array of, say, the embeds. Now once you have the embeds, we print these embeds and check the dimensions just to verify that. And we convert it into a tensor, which is using TORCH tensor. And the torch or tensor would be of our embed. Sorry, underscore array. Yep. So you should be able to get this result where you are on 10,000 documents. And. And each of them is a representation of 1024. Sorry, it was a 1024 representation of oz9. Sorry about that. We are using the light Model and not the English embed model. That's it, yeah. So once you get these, what, I'm sorry, I just remove these so that we could later use it for. Yeah. So now once we've got these embeddings, we use a dot product to check how similar one embedding is to the other. And in case of a semantic search, we have something called as a query and documents. So documents is the list of the entire embedding archive that we have. Like these, the all the product names that you have present in your stock. And query in our situation is say the item that has gone out of stock. So over here we have a data set that consists of toys and some goods. So we'll be using this one particular toy where we have it like Marvel Black Panther Legend series and it's of some 12 inch dimension and that has gone out of stock for a particular client and the client wants say another Marvel Black Panther branded toy itself, but with similar dimensions. So that we consider is our query. Now what we need to do is we need to embed these queries first of all. So to embed this query, we'll be using the same embedding method, which is let's copy the embedding, Paste it over here to replace these product names. And since our query is a string, we need to pass it in as a list because the text accepts a list of strings. Query. Yeah. And we do the same end embeddings and we convert into a torch dot tensor tensor of the query embeddings that we have. Yeah. So now that we have these tensors, we calculate the dot product between the query embedding and all the other product embeddings to see which one matches and which are some of our top dot products that we can achieve. To say that these are the products that are the similar alternates to the product that have been out of stock. So we use a torch or matrix multiplication between the query embeddings and the embed tensor and we retrieve the top five dot products that we have, we execute that and we then execute this. I'll just hide the result just in case people might want to execute it. And over here what we do is that we retrieve the product names of the top five of the top five similar products with respect to our reference product. Let's print it out. Yeah, you could just ignore the first, I could say the first document, the first recommendation, because that's the same product, since that's present in the data set as well. And now you can see that you have these four products that are there. So that way you have built, you know, a semantic search engine where you have an out of stock product and you're able to leave similar such products just using the Cohere API without using a lot of other, you know, technical functionality than having to know a lot about machine learning as such. Richie, any questions? Does, is anyone stopping?

There are some people saying they're having problems with the trial key here. So I suspect is the solution to this just to create a new trial key? If you're hitting the rate limit, yeah.

It'D be better to create a new trial key.

Okay, yeah. So there are instructions near the top of the notebook. So basically click on that link, the dashboard cohere aa/API keys. There should be a button there saying create a new trial key and then you can save that in your, in your workspace, environment variables. So if anyone's struggling with this, instructions are at the top. Try creating a new trial key. See what, see where you get to. We do have quite a lot of questions for the audience. Perhaps we'll go through them at the end. Just, we just need to make sure we got 10, 15 minutes or so at the end to go through all the questions.

Oh yeah, sure, yeah. So now that we have the semantic search that's been built, as we can see, we have these demand 6 inch scale or 12 inch action figure. We would want these to come up on top of our recommendations as compared to the other items which are, which not have a dimension or which, which are not close enough to the dimension of an item that has gone out of stock. So this is just an example where your four items could do it for like 10,000 items, 400, 500 items. And it should be pretty easy to do it just using the embed API that way. Now we come to another exciting part which is the information extraction. So now over here what we would want to do is we would want to extract information from our input text. And that too, we do this just using natural language without instructing the model using any sort of technical code. We would just instruct it in a way saying that, you know what, pass me a sort of a dimension from this text. So that's another cool magical feature that I think is really useful, that people can just use it out of the box and they don't need to have a separate sort of database where you can annotate stuff, or you need a separate database to store a lot of these separate attributes. I could say saves up a lot of your memory And a lot of maintenance on those to do that. We aren't going to use any other API right now, but we'll be using the same Coda Generate API. The Coda Generate API, as you all have seen, can be used for creative purposes, but it can even be used for these information extraction and entity extraction. As such, if you try to do this in without using say a Coda Generate API, what you have to do is you have to build a regex method in order to understand all sorts of dimensions in order to understand all sorts of different formatting that are being used for the dimensions, which could be quite time consuming and you might not be able to take in some edge cases and it might be frustrating at times due to the upcoming developments may be in the catalog of your product. So we'll be moving forward with this. And the Coda Generate model is going to help you extract dimensions right now from these descriptions. And these are like the four different ways that we are going to work with it. First is we're going to prepare an example for prompt which means that an example of a prompt I'll come later to it will create the extraction prompt. We'll get the data that we have already and we'll run the extraction function on this data. What do I mean by prepare the example for the prompt? What this means is that you have a sort of text but you cannot just go and tell the model, you know what, just extract a dimension. It's like telling a child without learning anything that go and do something, go and give sort of an example, go and you know, do a sort of a task that is not that it does not have an idea about. So what we do is we provide the model with some sort of example that you know what, this is the task I want and this is the sort of extraction I want. So we provide it with some sort of examples like this is a particular product description or product name and from that I want to extract 41 inch, which is similarly we provided some sort of example so it understands what is it that we actually want from the prompt that we pass it in. And as you can see over here we have another product name. And over here in this product name we have 1000 inch that has been provided as a dimension that has been extracted. In technical terms this is called as few shot prompting where we provide in some examples of some of the tasks that we would wanted to achieve over multiple different instances. Let's do one thing is let's initially paste this dimension examples which is a few short prompts into our code here and run that. Once we run that, what we end up doing is we end up creating these sorts of. We'll have to end up. We end up creating these sorts of examples and these examples will then be concaved with our input, which is all the alternate products that we need to get the dimensions extracted from. So I'll give you an example of what we are trying to do over here is we are trying to create this sort of an extraction prompt where we have all our examples and we have the other example from which we'd like to get our dimension. And for that I've created this complex, this class which is called as a dimension extractor. The dimension extractor takes in the example, which is the product name, the example labels, which is the extracted dimension, and the example prompt. And the example prompt is something, I'll come to it later as to what it means. Then we have a function called as make prompt which actually ends up making the prompt, which is the text that gets passed onto the model in order for you to retrieve your dimensions. And we have the extract which uses the code generate function. And we have changed it to a maximum token of 10 and a temperature of 0.1 because we want a more deterministic answer rather than very interesting and creative answers. And we have put in a stop sequence of say, you know, a new line over here as an example. So let's just paste that over here and run that part of the code. Yeah. Now once we're done with that, it said that we have something called as an example prompt. Now what an example prompt argument does is it instructs your model, telling what actually are we trying to achieve. I'll give an example over here. We can go ahead over here and execute this one particular class that we have and initialize it. So this should help you all over here. I'll just keep it on for another 30 to 40 seconds so people can just get acquainted. Yeah. And so now over here what we do is we have this cohere mentioned extractor class. We do something called as a make prompt. And say we just pass in an example something called as an input text string. What happens over here is you can see that this is the way the input is being presented to the model. So you have some sort of context or you could say a preamble to it. And you pass in this sort of an instruction extract product description from product name. We had the label, we had the product name. And this is our example prompt. This over here that you can see that Our personalization is our example prompt. And we do that for these examples where we say that, you know what, this is a product name. Extract product dimension from product. Extract product dimension from product name and you give it the answer. So the model at the time of inference right now understands the fact that you know what, this is the particular context. This is what I'm supposed to do. And this is sort of result it wants. Similarly, what happens over here is you pass in your input over here input text, which I could say is your alternate products that we just searched. So I'll just take an example. Let me just go back ahead so that doesn't confuse you all. Yeah, I'll just take in this example over here so that you people understand what you're trying to do. And let's just replace this input text over here with this context. So over here what happens is this is the entire preamble or the text that has been passed on to the model in order for them to retrieve your dimension for your alternate products. So over here you can see that we have been. This is what we'll be passing, which is the Avengers, Marvel Black Panther 6 inch scale, Marvel superhero action figure. And we said extract the product dimension from this product name. So this is just the make prompt part of it. And this make prompt is passed on to your extract function. So let's go ahead and execute this particular loop where it takes every single dimension in your result and it ends up providing you your. It ends up providing you all the sorts of extracted dimensions. This might take a bit of. Yeah, so we have done that and let's. Okay, so here you go. So just through natural language, just understanding the natural language, it's able to extract 6 inch, 12 inch. And for these other product items that did not have a dimension, it returned back none. Which is a really amazing thing that happens where you just provide a natural language. You don't have to worry about any other sort of maintenance or any sort of other piece of code to take care of it. All you could do is just pass in your sort of instructions in natural language on what you want to do and what sort of information you need and you'll be able to extract it this way. Now that we have been able to extract these dimensions, we'll be moving forward to these products. Now in the interest of time, what I've done is created this function that is used to rank your product and provide you the most similar products based on the reference of your out of stock products. So all you could do is just copy this Product and paste it over here, copy the snippet and paste it over here and should be able to execute that. And once you're done executing that, you could execute this other piece of code. Sorry, I might have chunked it down a bit. Yeah, Yeah, over here. So you can see that we have been able to rank these items based on the most similar dimension for an alternate product. So for the product that was out of stock, which was a 12 inch laptop of black, of say an Avengers Marvel Black Panther item, we are able to get a similar product which is the Avengers Marvel Endgame, Titan Hero series, Black Panther 12 inch action figure, and then a 6 inch scale of some action figure toy that we have. So that way you've been able to create an alternate product search engine where you're able to customize it based on any sort of natural language inputs and just using an embedding search method that's there. Yeah. So I hope you all have understood and been able to enjoy through this process of understanding the Cohere API and how you can use the CO generate and the CO embed models for your particular purposes and can do all this in like less than, you know, like 200 lines of code and like less than 100 lines of code and everything in natural language and without the need to worry about maintaining your own sort of an ML system and all those other things. Yeah, thanks.

That Rishi, that was really good. I have to say I especially liked a bit about extracting information because that seems like it's really useful for data cleaning because quite often you end up with data that's just like a mess of text and a lot of it's nonsense. You want to make it a bit more structured. So that seems incredibly useful for that. We're going to go to audience questions in a moment. I have a question for you. Since this was like mostly product focused. Is that like the main use case for the Cohere API or other use cases that different teams or different areas of data might be interested in?

Yeah, there are multiple. So you can use it in fintech for extracting things from your documents. It's not just product. Product was something I thought of taking up because this was personally my own experience. So PSV where I worked, we didn't have Cohere API or we didn't have any sort of a model to do this sort of extraction. And I had faced the same personal use case where we had a client who had asked us to give us an alternate product that can help increase the profit margin and we didn't have enough data to get these sort of rejects and, you know, be able to get recommendations in terms of these sort of rankings. And this was a really difficult task. This didn't take me much time to, you know, execute and do it in just like, let's say, a day. So that's the reason why this was a bit close to me. And I thought of going into the E Commerce product side of it.

No, it's pretty. I mean, it's such a widely applicable use case and yeah, I can certainly see how some of you scale cases. You know, they take a lot of code and a lot more effort to kind of build up, but. All right, let's go to audience questions. So this question comes from D Analyst 007. A bit of a bond theme there. All right, so it's talking about the semantic search and how it works. Okay, maybe can you just go through, like, how the semantic search just finds. Does the ranking. Like, how's it going?

Yeah, yeah. So. Oh, you could do multiple things. Like. So semantic search, what it does is you have content, which. So you have a coherence model that has been trained on a huge number of content, and you have representations. Representations means that these are the numerical meanings of a particular document. So whenever you pass any sort of a query or any sort of a text to that particular model, it has gone through a lot of text previously and learned all from the model, has learned a lot that it knows what it means. Now once you have what that particular vector means in a numerical representation, we use a mathematical method called as a dot product. So dot product is technically used to find out similarity between two vectors because it takes into consideration considering the orientation and magnitude. And we use a dot product to find the similarity between the two items, and the one with which has the highest score would come up on top with respect to a dot product. So that is how you would use a semantic search for similarity purposes.

Okay, nice. Next question comes from Eliza. So Eliza asks, is the API multilingual? So we were just doing English content here. Does it work with other languages?

Yeah, yeah, definitely. Like, that's an offering we have as well over here. So if you could click onto multilingual search, you should be able to find a. I'll go to the document. Yeah, you should be able to find the document on how you could use multilingual semantic search over here. So we do have some multilingual embeddings as well.

All right.

Do you have a sense of which languages are supported there?

I'm not sure yeah, there are over 100 languages supported. So I'm.

Okay, I'm not going to make you list them all. Yeah, yeah, I imagine. Okay, so most popular languages support it. That's. That's good to know.

Yeah, yeah, I think it's 100. I'm not completely sure of that. So yeah, you can just check out into the multilingual semantic search document. I think if it's present in docs. If you go on to multilingual embed models here you have it over here where you can go further more into it. Yeah, yeah, we have over 100 languages. Yeah.

Nice, nice. Okay, we got one more question from the audience. So if anyone else in the audience, if you've got questions for Rishi, please do ask them in the chat. Now we've got a couple of minutes left. This question comes from Matt. So is it possible to extract information with a confidence parameter to gauge how confident model is that the response is representative with the examples you provide. Okay, so yeah, do you have some sort of measure of how good the answers are?

So currently we just have the likelihoods that we represent. We return. We don't have another sort of, you know, a confidence metric that we use to tell how good as such the answers are, but we could say based on the likelihoods that you have to say how probable the outputs are for you to be able to understand on, you know, the confidence or particular input that we have.

Yeah, okay. See I've got some score of just, you know.

Yeah, we have the likelihoods. Yeah, yeah.

Okay. Yeah, all right, that seems pretty good. Okay, so next question comes from another from Alex. So it's possible to use presidential's election data. Oh, there's another part of this based on sentiment analysis.

Oh yeah. So I haven't explored that personally myself on. So we do have restrictions with respect to some sensitive content. So you could try it out yourself and see if you're able to use it across, you know, like I think based on history it would be able to do but the model would not end up providing an opinion for you. So I think you should be able to use it. Yeah, but we even go through a sort of toxicity detection and to be able to understand is is there a sort of a bias or is there some sort of toxicity in the content before you do a sort of semantic classifications.

Yeah, imagine there's tons of social science use cases there once you like so much text data available. All right, brilliant. So before we finish up, I just want to talk about upcoming webinars. So on Thursday we've got a session on best practices for hiring data scientists. So this is particularly for managers, anyone involved in hiring. A lot of job interviews are a bit rubbish and so this talks about how you do do things better if you're job seeking. Might also be useful to learn a little bit more about the the interview process. We've got a break from webinars next week so I'm around and then we're returning again. But there's another section on generative, another session on generative AI. Next one is Wednesday, October 18th. Please do go to datacamp.comwebinars to sign up for those. So yeah, thank you once again, Rishi. That was a really great session. Yeah, I learned a lot. And thank you to Rhys for moderating. Thank you to everyone who asked a question. Thank you to everyone who showed up today. Hope to see you again on Thursday or in two weeks time. See you again soon. Bye.