Cohere's Command-R+ Specialized Model for RAG and Tools

Today Cohere released a very interesting model called Command R. It's a bigger sibling of their previous model called Command R that was released a few weeks ago. If you are not familiar with Cohere, it's a foundational model creator that is specifically focused on retrieval augmented generation at production scale. So this is very different from other companies like OpenAI, Anthropic and Mistral because they are simply focused on solving retrieval augmented generation tasks for enterprises. So just like the previous version of Command R plus, this model has very strong accuracy on retrieval augmented generation tasks and it can also use tools. It's supposed to have low latency and high throughput and has a much larger context window of 128,000 tokens with comparatively lower pricing. And it's also multilingual, so it has support for 10 different languages. So in this video we are going to look at some of their results and then we will test this model out ourselves. For their previous version of Command R model they also provided this needles in the haystack evaluation where it performs almost perfectly. The now the Command R model was a 34 billion parameter model, whereas the plus version is 104 billion parameters. So it's much bigger and it also shows up in the relative performance. So here they're comparing the Command R model against mistral large and GPT4 Turbo for multilingual capabilities. It outperforms the Mistral Large model by a large margin. I think it's very close to GPT4 Turbo. When it comes to rag, we see a very similar pattern, but for tool usage they claim to outperform GPT4 Turbo, which is going to be very interesting to see. Now when it comes to pricing, it's definitely priced lower than the Mistral Large and GPT4 Turbo, but it seems to be priced higher than some of the other LLM providers. As a company, their focus has always been on RAG or Retrieval Augmented Generation and this new model seems to do pretty good when it comes to rag. According to their blog post, Command R is optimized for advanced RAG to provide enterprise ready, highly reliable and verifiable information. And this new model improves on response accuracy and provide inline citation that mitigates hallucination. So this could be a perfect solution for finance, hr, sales, marketing and customer support. Based on their internal evaluations, their results are also pretty impressive. So here they are providing human preference evaluation for summarization with citation and they compared it with Sonnet, which is the Cloud 3 medium version, and people tend to like this model 74 times compared to Sonnet, which is only 26 times and it does a pretty good when it compared to GPT4 as well. Now on multi step reasoning with search tools also, this tends to be doing pretty good when it comes to the standard benchmarks. It's just behind GPT4 Turbo. So in general I would say that these are pretty impressive results. Even for conversational agent and function calling, it seems to outperform the Claude 3 sonnet and mixed large model. Now something that is missing in Here is the Claude 3 Opus model. Since they're putting in GPT4, I think it was fair to also put the Claude 3 Opus model. So it will be interesting to see how it compares with the Opus model because Opus seems to outperform GPT4 on some of these tasks. Now this model is multilingual and it has support for 10 different languages including English, French, Spanish, Italian, German, Portuguese, Japanese, Korean, Arabic and Chinese. So it's very different from the other LLM providers that we have seen because it has a support for very diverse languages, specifically something like Arabic or Portuguese. And even on the multilingual evaluation tasks, the performance is pretty great when compared to the other LLM providers. So I think when it comes to enterprises that are serving customers in different regions with multiple languages, this is definitely a model that enterprises are going to consider. And for the multilingual token cost, this has a relatively lower token cost. When you're comparing with Mistral and OpenAI offerings, the models are not open source even though the weights are publicly available. So I think it's a very interesting strategy. They want people to experiment with the model weights, but they're not allowing them to use it for commercial purposes. In order to use the model you will have to use Coheres API. And here are the pricing if you want to experiment with the model, they provide a platform called Cohere Coral and you have three different options here so you can chat directly with the model. This will generate responses from the model. Then you can augment the responses with web search and the last option is Correlate with documents. So basically you can do retrieval augmented generation on your own documents by uploading your documents in PDF or text format. In this video I'll show you Coral with web search and Coral with documents. So let's first try out the Coral with web search. Okay, so as to what is Jamba LLM? Jamba LLM is a new architecture that combines the MAMBA architecture with transformers and it's actually able to do web search correctly. So it says Jamba is state of the art Hybrid SSM Transformer LLM developed by AI21 Labs is the first production scale MAMBA implementation combining the Mamba structured state space technology with elements of the traditional transformer architecture. And it's actually correct. Now the good thing is that it provides inline citation. So let's say if I click on state of the art here, it will highlight different web sources that IT use and it will also highlight the keyword. So let's click on this SSM transformer. You can see that it is highlighting those sources. So this is pretty neat. So it kind of gives you a sense of using Perplexity AI. Now the great thing is it also suggests subsequent or follow up questions. So for example, here it states what are the benefits of using JAMBA LLM? Let's see what response it generates. So you can see that it is doing web search and specifically it's doing web search for benefits of using Jamba LLM. Next let's look at the Coral with documents. So here now you can enable web search. So this will basically augment answers both from your documents as well as it will do web search for generating answers. But in this case I want to just chat with the original ARCA paper and my question is going to be what is instruction tuning? Now this does take some time, so it's slower than the web search because it has to do RAG on the document that we provided. And the answer is instruction tuning is a technique that allows pre trained language models to learn from input and response pairs. Now again you can click on this and this will basically tell you the reference. So the reference is coming from the ORCA paper and it highlights specific section of the documents from which it was getting those specific segments. And here again it was also suggesting some follow up questions. So in general I think it's a very interesting model that has some real practical applications when it comes to enterprise use cases. And that's exactly what Cohere is going for. Now this model is not available as an open source model, but it's great to see that these model providers are training task specific models with very improved capabilities. So if you have a business use case that relies heavily on rag, I highly recommend to check this model out. Let me know what you think about the model. I hope you found this video useful. Thanks for watching and as always see you in the next one.