The AI Skill That Will Define Your PM Career in 2025 | Aman Khan (Arize)

Evals actually force you to get into the shoes of your user. Like you can no longer just hypothesize, I wonder if they'll do this or I wonder if they'll do that. Like it kind of forces the PM to be way more empathetic of the customer and what the customer is going to try to do and articulate that in writing, which is a whole nother skill. So you want to translate your understanding of the user into text and then you want to kind of use that to iterate on the system. I think that's hard. I also think that, you know, there's a high level of subjectivity and non determinism which makes it challenging. Like it's not as easy to predict as writing code, like what the outcomes are going to be in this case. And you're trying to model human behavior as much as possible. Let's say the LLM writes a sentence or the AI writes a sentence, like, was that a good sentence? It depends a lot on the context. It can depend a lot on the different scenarios that you're putting the system into. And so understanding and looking at the data and basically seeing, okay, here's the clusters of types of scenarios that the system is going to be put into and being. Being able to break the system up into more components and more fine grained scenarios makes the writing of the eval easier because now you're, you're iterating in a tighter loop. But it is really hard to think about this when you're first getting started.

All right, welcome everyone. My guest today is Aman Khan, Director of Product at Arise AI. So Aman and I attended a talk recently. Chief product officers of OpenAI and Anthropic, who both agreed that writing AI evaluations will be the most important skill for product teams moving forward. So I'm really excited digging with Aman today on the key skills that AIPMs need, including learning how to get better at writing evaluations. So welcome Aman.

Yeah, awesome. Thanks so much for having me, Peter. And yeah, that was a pretty crazy moment to be sitting in the audience there. And then I remember we both like left the talk and started talking about evalves and yeah, really glad we get to do this.

Yeah. So maybe you could start by introducing yourself and like how you became an AIPM at a company that specializes in writing evals.

Yeah, for sure. So funnily enough, I've actually been kind of working on the same problem for most of my career, so I actually started by working on evals for self driving. So Actually started as an engineer at a company working on in the self driving space and then kind of realized that it's really messy trying to figure out what exactly do we need to do to make the car better or worse. And this is super applicable now even in the LLM and generative world where you're trying to work with these black boxes but you're not really sure am I making this thing better? You know, how do I measure the performance of the system? And I kind of just kept that through line by, you know, realizing there's actually a lot of products to be built in the space, a lot of, you know, really messy like dashboards or getting the right metrics and right data in place. And so that actually allowed me to leap to going to Spotify to work on the machine learning platform. And while I was there I was sort of still working on the same sort of data problem. How do we use data in machine learning in the models that we're shipping in production. And then it was actually there that I figured sort of actually discovered the company I'm at now called Arise. And Spotify is actually a customer of Arise, which sort of worked out in a funny way there. But I sort of just kept that through line going of trying to make it easier for people, engineers, product people that are building with AI systems understand the impact of their models. And I think this is going to be a pretty long tail problem. So a lot of really interesting product challenges along the way to solve.

Yeah, because you know, these models are like a black box. You can't really tell what's going on. But you know, like everyone keeps talking about aipm. It's kind of become a meme, this AIPM thing. And you know, everyone wants to become AIPM these days. So if you had to list like a couple of key skills that into you can build right now to become an aipm, like what would it be?

Yeah, so it might even be helpful just to like sort of recap for folks that, you know, if this term is new, just in case, just to like set the right terminologies if defined correctly. But I kind of view AI PMS as people that are building products with or around AI. And that means, you know, it sounds silly to say now like I think that AI feels very new. It's like everyone's trying to catch up. But you know, ultimately I do think long term all PMs are going to be some flavor of AI PM. Either you're building on top of some sort of LLM API doing something in or Some model in the back end performing some action in your product or you're going to be using these tools in your day to day, you know, to solve problems. And so I think it's helpful to sort of frame this as really AI starts to become a core part of the skill stack for PMs in the first place. And then once you kind of start from there, I think there's maybe, you know, like kind of a few sort of high level ways to think about how to build your own knowledge in the space. I sort of look at like maybe like five different areas here. So I would kind of view this as, you know, start with the fundamentals. What are the high level areas of AI and ML that you know, you would really are being used today in the industry to actually deploy these models in the first place. And think about this in the context of what would you want to apply in your products or in the problems that you're trying to solve for your customers. There's a couple helpful resources. I sort of plugged the Andre Karpathy one in the last podcast I did with Lenny, but I think another really helpful resource I've kind of been referencing myself and came across is there's a really nice algorithm map that Merilee, who's an AI product manager as well at Google, has put together where she actually goes through. You know, here's when you would use this algo for this thing. And this was. This is what machine learning is useful for. I'll give you an example. Things like, you know, understanding. Okay, what do I do to predict a user's actions? Should I use a regression model, a classification model? It helps you frame the decisions that you want to make in the form of statistics and models. And I think that's just a helpful framing to have. So basically the goal here is understanding what these things are good at and what they're not good at. And that's really the core intuition that you're trying to build. I ironically. Yeah, I'll pause there in case there's any thought.

No, I think that's a really good point because it's important to realize that there's actually different types of models, like you said, that solve different problems. Like a model that solves discovery is very different from like a LL model, right?

Totally.

Yeah, totally.

Yeah. You can't just like, it's like showing up to a hardware store and you're like, you know, I need a tool. And the people are going to be like, what tool do you need? And you're like, you can't just use a hammer for everything, right? Like, there's that expression that goes like, if you're a hammer, everything's a nail. It's sort of the same thing with AI. Like, if you're using LLMs for everything, you're probably going to have a bad time. These things aren't necessarily great at everything. You might need a different algo, you might need a different approach. They're very good at generalizing. But it's helpful to understand how to think about the space more broadly. So I do think that fundamental starting point is really important. And you might actually find as you're going through this journey that certain algorithms or certain spaces of machine learning or AI are maybe more interesting to you and more interesting for you to go down that deep rabbit hole. It's very likely that, you know, people are already talking about an industry that we're sort of hitting a plateau with what LLMs can do. And so ultimately we're going to need to combine different algorithms to actually get the results that we want. And we actually see that ourselves as well with customers that we work with. It's not in LLM going and doing something. It's usually, like, combined with a ranking model to give you a result or a classification model. So that's the foundation. I'll kind of go through a couple more areas which I actually think are core product skills. In the first place, customer obsession and curiosity are really the two things that are going to drive you kind of coming back to, okay, now that you have the foundation, you have at least the basis. You're not going to have a master's or PhD in the subject, but at least, you know, okay, this is what this thing is good at. This is what this other thing is good at. Now you kind of let your curiosity drive you to the next stage, which is, you know, what are the problems I'm trying to solve for my customers? What's the thing that's most important to solve for them? And then allow you to really take that curiosity and keep trying to prototype and like, basically try and solve the problem for yourself. And there's sort of this, like, thought where, you know, I, I think that prototyping is going to be a pretty core skill. I think that curiosity is going to pivot into prototyping and just being able to try the tools yourselves. You'll learn from, you know, the AI experiences in the first place, what is useful in your day to day, and what's like a terrible tool in the first place. So you're kind of like prototyping, developing that intuition. And then I do think last but not least, okay, you've got the foundation, you've got the knowledge, you have the understanding for what to go and solve for your customers, and you know how to build prototypes. You're not done there. You actually do still need to understand how well your system is performing overall. And that's really where evals and observability come in. And I do think that's going to be maybe one of the most important skills for PMs be able to learn is you can't just ship with AI. You have to be able to measure what your system is.

Got it. So can you just like quickly recap like the key skills again?

I kind of, if I had to boil it all down, I would say the fundamentals, customer obsession, curiosity to learn and prototype, learning from great AI experiences and then eval, kind of make up this like five pillars of what you'll need to do or need to learn as an AI PM yourself. And if I were to put it into one sentence, I would say build knowledge, experience and best practices on shipping high quality AI products in the real world and then be able to measure the impact. Got it.

I actually think like the curiosity part is actually pretty important. I think like a lot of big company PMs actually don't have that curiosity or they're like too busy just like riding slide decks and like making dogs and times. You actually pay around and stop themselves.

Exactly. I mean, I think that's like, that's sort of the realization in myself as well. Like, you know, imagine you could just show up to the meeting with a prototype instead of PRD. I think that's the shift that PMs are going to have to undergo as they figure out how to communicate better in this new world.

And, and also I think it's important to emphasize, I think that you don't have to add this. You don't even have to have PM as a job title to start working on the skills, you know, because like AI has made it. It's actually arguably easier to ship stuff with AI yourself than to try to get buying out of company.

Yeah, I think that there's like a. It's funny, like, I'm sure you've, I've seen that you've like made fun of this stuff in the past. But like, there's always that like, trope of like the PM is the CEO of the product, you know, like that's what people say, oh, the CEO of the product. But like, I actually think that has never been true until now because now you actually have these, like, superpowers that you as a CEO of a product can actually go and be like, let me go write some marketing copy and test it out. Let me go build a quick version of the website. I don't have to wait for the engineering team to do this. Like, you can just go and start getting things done. And that can really be motivational for the team. And I highly encourage that's, like, where you'll learn. Okay. I actually, you know, this is where I need help. This is where we need to iterate. So actually, you can be the CEO of the product now. I think before was like, you know, people were just saying that to feel better about themselves.

Yeah, yeah. And like, you know, when people think AI tools, AI products, a lot of people just go naturally towards the chatbot. Right? Just make another chatbot. But, like, you know, I think a lot of the really good AI products are actually not very sexy. They're like, they're more focused on, like, fixing tiny little friction points in the core workflow. So I'm curious, like, outside of like, you know, chatgpt and like, you know, cloud and stuff, what are some of your favorite AI tools in the market around this?

I think lately I've been pretty obsessed with the prototyping tools out there and, like, coding agents. We can plug, like, a Twitter thread where I actually break down. There's a lot of them out there, so it's a little bit overwhelming. I spent a weekend just trying a bunch and then, like, dumping my own thoughts. And I've just been using those a ton for prototyping. I'll give you an example. So we were sitting in a meeting and we were like, darn, we really need to, like, ship this usage dashboard for a customer so that they can see, you know, across their enterprise how many people are using the product. And we're going to need to put, like, an engineer on this for like, three weeks or three days or a week to like, go and figure out, okay, here's how we pull the data in the right place and then, like, build a nice ui. And then, like, while we're debating all of that, I literally just went to Replit, and Replit just lets you type in a prompt. It's not really a chatbot because it's not going to give you a bunch of text. It's actually going to go and write code for you to actually build a real functional prototype. And I literally just did that in the meeting. I copy pasted the Pendo Docs, which is a tool that we use. And I basically told it, okay, build a usage dashboard for this customer. Here's an API key. And in the end of the meeting we had like something we could look at and be like, oh wow, we just did that while we were talking about it. So I think that bias to action is going to help a ton, but I'm obsessed pretty lately with the coding agents. Another example is maybe like two more examples here actually. One is that, you know, I needed, you know, a quick refresh of my personal website. And I still remember the first time I coded my personal website, you know, after HTML then I wanted to like do something a bit more like React native. And I took like a week in college to go like at the end of my nights and weekends like going, right, oh, this is, you know, like my personal portfolio, whatever. And then you like, you have like the WIX templates, but you don't really like those. You want it to be a bit more personal. And now with Vercel, there's a tool called V0 which builds really beautiful front ends and so, and you know, it does more than that as well. It's pretty, pretty capable product. But personally I could look and feel of that and you can just like drop images in and get, you know, pretty functional prototype result at the other end of it. So I use that to build my personal website. And then I think last but not least, I've been playing around a lot with like Claude writing styles. So, you know, you can actually give a sample of your writing now and Claude will try and build a, you know, sort of, you know, actually sort of mimic the way that you write based on that. And it's pretty good. It's not fully there yet, but it is the cold start problem there a little.

Yeah, that, that's what I've been doing with Claude, just like giving a touch on my really good writing and like, you know, being lazy, asking to edit stuff for me. Yeah, it works out well. Yeah, yeah.

And then the skill just starts to become how do you get the right product out of that? How do you prompt it the right way to actually get the result that you want?

Yeah, the prototyping thing is interesting. Like I also try to play around with some of this stuff. I feel like it's pretty good for making like little toy apps and like promo concepts for yourself and maybe a personal website too. But like it's not, it hasn't got to a point where you can actually like make a SaaS product and sell it all alike.

But, well.

But I feel like it's pretty close. I don't know. I feel like a year from now it could happen.

Yeah, Yeah, I think that's totally true. Like, what's funny is that, like, if you go on YouTube, there's, like, so many videos. All you have to do, like, a little hack.

If you.

If, like, to anyone listening to this and, like, wants to try this out themselves, but they're like, you know, maybe they're sitting on their sofa, they don't want to, like, go reach for their laptop. Just go to YouTube and, like, type in, you know, cursor or V0 and you'll see a ton of tutorials where people will, like, build a SaaS app in, like, 10 minutes. A lot of people are claiming to be able to go and build entire companies on top of that. I don't know how feasible that is necessarily, but it is really interesting to see that's where the space is headed for. Sure.

Got it. Yeah. I'm not sure I believe people claim that they're making a lot of money from this stuff, but we'll see. But just to wrap up this chat, I feel like what I really love about these AI tools is that there's so much time wasted by PMs to even try to get a job as PM. You have those case interviews that are kind of like, hypothetical. And then you got to do, like, in your PM job, you're writing all these docs, you do go through all these reviews, and none of the shit actually about the real product that you're trying to build.

Right? Yeah.

So I hope with this AI tools, we actually shift attention back to the actual product trying to deliver to customers. That is my dream.

I think we're getting there closer. And you were at the same conference and there's this talk from Clear Vo where you know, just like, Venn diagrams of, like, functional areas and jobs. And like, I do. I actually do believe in the thesis that, like, jobs and roles are going to be less important and actually skills and how you use tools to your advantage are going to be more important. So that's like an example where, you know, you. You're not just going to be writing docs. I think pretty soon I think it'll be a lot more about, like, actually solving the end user's problems even more. So.

Yeah, that hopefully make the PM job a lot more fun.

Yeah, yeah.

But, yeah, let's talk about eval. Let's talk about, like, the most important skill, man, for pms. So so, yeah. Well, what exactly is an eval and why is it so important?

Yeah, so I think it's helpful to frame this a little bit as well, like in the context of a product. And I'll make it really simple. This is my mental model that I kind of use to think about building products in the first place. I think if you really boil it down, a product, imagine like a, you know, a diagram. A product is a box in the middle, and you have an input and an output. And the input could be things like, you know, a customer coming and trying to solve a problem, and the output is the solution. Like the product is trying to solve that, you know, in some way could be like, let's make a tactical. Let's say I'm going to a website and I want to, you know, have like, and, you know, basically have this experience where I type in, I'm going to, you know, San Francisco and help me book a trip to do that. You know, help me book everything in the middle. And the outcome is you actually get your trip book. Now, there's a lot of steps in the middle. Like, we were oversimplifying this, but at the end of the day, what you care about is that the user has a good experience and evals are the way that you measure how good or bad the box in the middle the product is. And I think before, in like, software, you know, 2.0, you know, whatever software 1.0, 2.0, like, days, you could kind of view like, okay, are we calling the right API? Are we, you know, is the code being constructed correctly? Like, this is a very deterministic system. And now the entire system is a lot less deterministic because of large language models. These are statistical systems now. So there's a certain degree of, like, variability, and there's work being done to make the systems more reliable and just reason better and think more, you know, like, you know, OpenAI released this just yesterday. They have a much more reliable model called 01 Pro, but it's much slower. And so, you know, really what you need are tools that measure the goodness of the system. And those can come in a few different flavors that we can get into in terms of, like, code feedback and, you know, using other LLMs to actually measure the performance of the system as a whole. So that's a high level of, like, eval. Evals are how you measure the goodness of the system. And that's really consists of your data, your eval method, and, you know, basically the metric you want to use to actually analyze the System as a whole.

How good is the output given the input?

Right.

Like, how good is the black box? I'm making the output given some user input.

Exactly. How good is it at solving. How good is your product at solving this problem qualitatively and quantitatively?

Yeah. I think the difference is, like, in the good old days, the user only had one type of input. Like, you click a button and then it does this. But now the user has thousands of different types of inputs.

Definitely. And it's like these systems are becoming like multimodal. There's voice that they hear you correctly, you know, and even then, I. I would actually even argue, like in the product, you know, PM days of like, oh, is the user clicking this button? Like, now you can get much deeper and understand, like, why are they, you know, doing something or not? You can get a lot more. The data becomes much richer to use for debugging with LLMs.

Yeah, that's a really good point because the user is actually maybe telling the problem through chat or something, as opposed to clicking on button. Yeah, it's good.

Exactly. Yeah.

So can you talk about briefly, like, where do evals fit in the process of building a AI product?

Yeah, so I was kind of giving this analogy to someone, like, imagine that you're like, going to study to become a doctor. You know, like, you. You want to be. Yeah, you want to go be a doctor. Like, imagine that's the product that you're trying to build is like some sort of like, doctor or something like this. The eval is basically the test that you're giving the doctor before the doctor is actually starting to see patients in the first place. You. I mean, there's a ton of reviews. You want to make sure that the system, like, understands what it's getting into. Like, you know, there's like, technical things, you know, scientific things you're trying to, like, solve for to make sure that this system is actually good enough in the first place before it starts getting into production. And an eval is. Sits at that earliest step. It's like, think of it as like the medical text or the test that you're giving the system. And you want that to be representative of the real world. Like, it doesn't make sense to, like, test doctors on, like, theoretical things that they're not ever going to apply. So you really want the eval to be sort of at the earliest stage of development, what you're using to guide the system as a whole. So once you've identified a problem and you figured out, okay, here's what the inputs look like, here's what it should do, and here's the output that we're going for. The eval helps you measure at the earliest stages of development. You know, are we actually building the right system so that, that's really where eval starts.

Okay, so is it kind of like analogous to like running a, a B test or something? Like, you know, or maybe I guess AB can be part of evals.

Yeah, yeah, that's I think like it almost feels like evals are like a new kind of a B test. Like before you would have to, you know, split your audience and like get into production and see like what are certain people doing, you know, and are they like, what do they like this color or that color better? Like evalves are sort of the way that you do that with LLM based systems and eval sort of tell you. Yeah, they're just this new flavor of a B test in a way. It's a good way to put it.

Okay, so let's talk about the sausages of me a little bit. Right, so like. Well, first of all, I'm not an expert on writing emails, but you are. But when I write emails I, you know, usually start the manual process. Right, so like you like, you like have like what are five of the most common questions the user will be asking and then you try to write, you try to write like, okay, here's like the five ideal ad ansells and then it just feels like, it honestly feels more painful than like trying to write a TRV because you actually have to clear yourself in a user shoes and think about ads. But like why is it so hard, man? Like why is this or is there ways to make it easier?

Yeah, yeah, that's a great question. I mean I think that you bring up a really good point which is evals actually force you to get into the shoes of your user. Like you can no longer just hypothesize, I wonder if they'll do this or I wonder if they'll do that. Like it kind of forces the PM to be way more empathetic of the customer and what the customer is going to try to do and articulate that in writing, which is a whole nother skill. So you want to translate your understanding of the user into text and then you want to kind of use that to iterate on the system. I think that's hard. I also think that, you know, there's a high level of subjectivity and non determinism which makes it challenging. Like you just, it's very it's not as easy to predict as writing code, like what the outcomes are going to be in this case. And you're trying to model human behavior as much as possible. And so when the system is subjective, that can be really tough. I can give an example from like the self driving space. Actually that's very, you know, analogous to here, which is when we were trying to solve evals for a self driving car, we had a bunch of, we basically have like humans sitting and saying like, okay, did the car make this left turn correctly? Was it a good left turn to make? Did it, was it smooth? Did they, did the car commit at the right time? Was it too fast, whatever it might be? And it turns out that's actually pretty subjective. Like you know, someone's, you know, good left turn is different. Like some people would be like, no, I would never have done that left turn. And then when you start adding in more variables, like imagine someone is like crossing at the same time on the sidewalk and the car makes left turn left. Should it have done that? It can get really specific and nuanced. And you know, I think that's actually really relatable to the LLM world now because it's hard to say, like let's say the LLM writes a sentence or the AI writes a sentence, like was that a good sentence? It depends a lot on the context. It can depend a lot on the different scenarios that you're putting the system into. And so understanding and looking at the data and basically seeing, okay, here's the clusters of types of scenarios that the system is going to be put into and being able to break the system up into more components and more fine grained scenarios makes the writing of the evals easier because now you're iterating in a tighter loop. But it is really hard to think about this when you're first getting started. For sure.

Got it. Yeah, you got, you got to like think about all the edge cases, think about where it could go wrong.

Yeah, got it.

Okay, so let's make this more practical. Let's take like a real example of AI. So like, you know, I think a pretty common AI product is like a customer support AI agent. Right? Like it helps you answer any customer support answers. And how would you use different types of evals to make this thing better? Yeah, yeah.

So like the, the sort of customer support, like classic, like you know, I'm trying to get a refund or something like this, like that type of like customer support agent.

Yeah, like, like for like a retail or something. Yeah, yeah.

For retail. Yeah. So I think there's a couple ways that you, you kind of want to think about the, the system. A lot of it comes down to the data itself. So how do you maybe to like reframe your question? Your thought is like, what are the ways that I can measure my system as a whole, right? Like how do I know if this retail agent is actually like solving the customer's problem? So an example is like I want to return, you know, this, you know, this laptop or something like that. Well, you actually, let's think about the system a little bit more deeply. You actually have a few things you can measure. So let's imagine it is a chat box just to simplify the system even more, just to get started. And you can say, okay, well there's one user that's typing in, I want to return my laptop, right? And then the agent can like perform some actions, you know, maybe reason about, okay, like are you in the, within the return window? Did you, is the laptop undamaged? Do you, are, you know, able to actually return this thing? And those are all sort of rules. Those are like, you know, those are things that the, the agent can kind of, the, the customer support agent can be, have like coded into it. And so that's pretty straightforward in the sense that you can check, okay, you know, you have some inputs from the user and you're transforming those into the rule based system that the agent needs to follow. So that's kind of like the high level where you have real user data and you have the out the LLMs outputs and you can kind of compare where there's actually some ground truth. For instance, I can say, no, I'm not within the return window. If the agent says, okay, well let's continue to process your return. Like that's not going to, that's an example where the agent has gone off the rails. It's hallucinating. And you can test that by actually having another human grade the responses and say this is an example where the agent was incorrect. So that's your immediate starting point, is just getting the right data in place, the input from the user and the human label. I think you can also use some pretty straightforward, simple classifications or guardrails. Like if I start swearing in the chat, that's a string match. That doesn't need to be a super sophisticated system. But you can also say, okay, is this customer being rude? Do you want to be able to detect that is the, are they trying to jailbreak my system or try to get it to say something? It shouldn't do, which happens sometimes. That's made in the news. So that's another example of where you would want to be able to check the system is not being manipulated. So that's like a high level where you really want to start getting started with the data. And then when you actually, you know, have like those first five or ten examples, you can use that to actually generate even more data to for testing. So you could try to simulate more scenarios where the LLM might be incorrect and try to make sure that your system is robust against those different scenarios. And then lastly, last but not least, you want to actually test this. So you go out and you test with beta users and you do the AB test like you were saying. And you might be testing different prompts against each other too, or different tools, or different parts of the system that you are editing. And you want to compare that to the original sort of examples you were using in experiments and see, okay, how far off are our experiments from the real world? Do we need to make our experiments better? And so this actually starts to look more like a iterative loop where you start with the data, you kind of build on top of that in different metric, with different metrics, you get more and more data, and then eventually you use the real world to actually improve the system as a whole.

Okay, so just so I understand it, so step one is you have a bunch of like, maybe you upload your return policy to the AI and then you have a bunch of pass real human agent conversations with customer support agents.

Right?

So maybe that's the data that you start with. And then maybe you write a few Groucho's answers for different stuff. Okay. And then there's like easy ways to filter out sort of rewards or like things that are completely not related to the customer support. That's step one. And then you actually get some other LLM, you get some other to generate synthetic data set based on your human data set and maybe like get one LLM to evaluate another LLM. Right, Exactly.

Is that right? Yeah. So you're kind of building this to double click on that. You're using the initial human labels as the way for you to seed is this, you know, how do you augment or automate that part of the job with an LLMs that you don't need to have a human labeling every single, you know, response. Once you actually ship the product, you can have another system sitting on top and doing that.

Okay, and then, and then we get to a point where like, okay, this is actually good enough to test with real users. So then you run an A B test or maybe like, yeah, you run an A B test with like maybe like 5% goes to this AI thing and 5% still go to the real human agents. And then you compare some sort of metric or like what are some metrics for the saying that you would look at like accuracy or.

Yeah, so I think, I think it depends very highly on the system that you're trying to build. So you give a good example of like customer support agent. There's you do want to check like accurate like the rules. Like is the LLM following the rules? That's accuracy. That's a pretty simple like comparison. You can also do if the LLM has to give more open ended responses and you want it to the tone of your company, you want it to be maybe extra professional if it's a legal company.