1:00
Route simple queries like PDF retrieval to cheap models and complex reasoning to expensive ones. This skill-document mapping approach cuts LLM costs by 30 to 50 percent while keeping performance where it matters.
Route simple queries like PDF retrieval to cheap models and complex reasoning to expensive ones. This skill-document mapping approach cuts LLM costs by 30 to 50 percent while keeping performance where it matters.
User in this particular agent, there may be a different— we call utterances. Utterance is nothing but a user query to the agent, right? We can classify this query as simple, medium, and complex. So sometimes the mistake that what I'm seeing in the market, people fix a model no matter irrespective of any user query, it is always going to some expensive models. Rather, we can classify as simple questions, go to the simpler model. So that's going to cost like 50% cheaper than the other premium models. For example, only if you have any complex query where it requires some reasoning, then go to the complex model. But if it's a simple query, that means you already have a PDF somewhere in the database, somewhere stored. Now user query is show me, show me, show me my PDF. In this case, you, you just, we just know, we just don't need any reasoning. It can just go to the very simpler model. So their cost is 30% comparing to the premium models. So that's another category that based on the skills, basically we have the skill document based on the query. It's all mapped to every skill document. If it's a simple skill, go to the simpler model. If it's a complex skill, go to the complex model only when it is complex. So this can cut down, I mean, a lot of cost
