Braintrust and Box on AI agents and the future of AI observability

if I use an LLM to evaluate another LLM, is that really actually doing anything? And I think the answer is yes, because if you think about the process of maybe writing an essay and grading an essay, it's actually much easier to grade an essay than it is to write one.

Hi, I'm Ben Kuss, CTO of Box. Today I'm joined by Ankur Goyal, CEO of Braintrust, Today we're here to talk about not only what agents can do, but also how to evaluate what they're doing and how to make sure that they're able to work in production with things like evals, testing, observability, and so on. Ankur, thanks for joining us.

Super excited to be here.

Ankur, I think you have spent more time with AI evals than anybody that I know. Can you tell us a little bit about your journey about when you first came across the need for evals, and about Braintrust and about your journey?

Absolutely. So I started a company pre-LLMs in kind of like the stone ages of AI where we did AI document processing. So we would take like invoices and bank statements and stuff and try to find the structured data inside of them.

Very near and dear to our hearts here at Box.

Yes, yes. And somehow it's still a real, it's much easier to do it now, but it's still a really big challenge. And What we found is that every time we changed the models or the way that we use the models, the prompts, or even like the code around the models, we would break something. So for example, we had like banking customers and we might be trying to solve a problem for one of our invoice users and we'd improve the invoice quality and then we'd break one of the banks. And obviously that didn't work very well. So we had to get really good at preventing that from happening before we shipped new versions of our product. And this was back in like 2018, 2019. At the time we were trying to figure this out and there wasn't really any software that, that could help us execute this workflow. And so we built an internal tool and it really helped. Like we, I remember at one point we were arguing about whether or not we should use a new model and we were arguing for about 6 months and then we built this tool and like 2 weeks later we switched to the new model. I didn't think too much of it at the time, but then the company got acquired and I led the AI team at Figma. And this was right around late 2022, early 2023 when ChatGPT came out and there was suddenly a lot of interest around using LLMs. So we started building a bunch of really cool stuff inside of Figma and we had exactly the same set of problems. And so we built a very similar set of tooling. And that's when I realized that, you know, if I had to do it in both places, maybe other people have this problem as well. And so I talked to a bunch of companies that were early into building AI products and realized that actually a lot of people are starting to have this problem. And that, that's how Braintrust came about.

Awesome. And so, um, I think a lot of sort of when we, you talk about Braintrust or, or similar products, or when you talk about AI, we use this word evals, right? Can you talk a little bit about like, what does it, what is an eval and what does it mean? And also like, people are familiar, I think, with benchmarks and things like that. And, and why is it something that Google Trends-wise goes up over time? Like, why is this a topic that people who deal with AI and AI agents are talking about?

Yeah, so I think eval is a term that kind of predates this AI thing. Like, if you're considering procuring a new piece of software, you eval it. If you are trying out a new relational database, you benchmark it. And it's really the same idea. An eval is kind of like the scientific method. You might have a hypothesis like, if I change my prompt or if I use a different model, it will improve the quality of my agent. And eval is the process of actually testing that. And honestly, I used to work on relational databases and we had really sophisticated technology that helped us benchmark new versions of our product. And if we improve the query optimizer for one kind of SQL query, eval all the other SQL queries and make sure they don't break. It's basically the same idea. The only difference is that in traditional software, when you're evaling or benchmarking stuff, you're trying to understand how fast it is, whereas in AI you're trying to understand how accurate it is. And I think that's the main, the main difference.

That's very interesting. And I think you highlight an interesting point, which is this idea of accuracy is actually sort of different when you're dealing with AI than it is if you were dealing with, say, traditional software development. Can you talk a little bit more about like why is that different? Like of course in all software you like run a test and then it gives you a result. Like what's different about AI in this case that requires this new level of determining accuracy?

Well, just like traditional observability is very non-deterministic, so you might build an app on your computer and then deploy it and then suddenly it's actually slow in production. And if you decompose why it's slow, maybe it's because the hardware that you're running it on is different, or suddenly you have a burst of user traffic, or there's all these random things that can happen. The interesting thing about AI is that there's a lot of random things that can happen, but they actually affect the correctness or the output that's generated. So in traditional software, you might type like 20 lines of code, and then every time you run the 20 lines of code, the same thing is going to happen. In AI, random things happen. And so you need to use some of the same methodology that you would use in measuring performance and reliability and so on to actually in AI measure the correctness of the thing that you're building.

Yeah. And you mentioned the word observability, and I think some people who don't necessarily sort of build agents all day long, like they think more traditionally of like, of course we have observability, of course we'll know, um, you monitor when something fails or doesn't. Um, but you're highlighting here that like the idea of AI observability is actually different. And can you relate this back to like the term MLOps or AIOps or what some people would call like the traditional way, like you talked about where you were using machine learning and how that has evolved over, let's say, the last 2 years?

For sure. I think there's 2 fundamental differences with AIOps or AI observability or MLOps and traditional observability. The first thing is that traditional observability is designed around the business problem of reliability and performance. So you probably spend a good amount of money here at Box on observability. And if Aaron asks you why, you'll say it's so that, you know, our systems are up, they're secure, they're fast. In AI, if you spend money on AI observability and someone asks why, the answer is, you know, maybe in addition to those things, making sure that it's actually correct or that it works. In ML, people would often measure this idea of drift. And drift is when you see things happening in production that are different than what you anticipated and trained your model on. In the LLM and AI world, you're not training models, and so you're not necessarily drifting from what you trained the model on, but you may still be experiencing unexpected behavior. And that unexpected behavior is in some ways drifting from what you evaluated and tested on.

And can you talk about how this concept I think is inherently built into like generative AI is made, is different when you're using AI agents, in particular the agents that are more sophisticated, more complex, that loop and that kind of reason over more complex use cases?

Absolutely. I think intuitively every step of an agent is non-deterministic. And you and I have talked about this. Agents are getting super complicated. They're running hundreds or thousands of steps, generating giant traces of data. And so every single time one of those steps runs, you're actually compounding the probability that something might go wrong. And so not only is it highly likely if you're building an agent that something will eventually go wrong, because if it didn't, then the agent would take over the world. But the decision paths and the number of ways that it can go wrong also kind of explodes. And so you have to deal with complexity on both of those fronts.

Yeah. I've heard you say something, which is AI evals are the new PRDs. Can you explain a little bit about like what you meant by that concept and why that is something that maybe is, um, when you're dealing with AI agents in particular, like this is a really important concept?

For sure. I think fundamentally this thing in AI that I think we've all probably heard, um, it's super easy to build a prototype and then it's very difficult to make it work in production. Part of that, of course, is because now you can vibe code a prototype. Yeah, but I think a lot of it has to do with the fact that the user interface of an AI product is very simple. It's often just a chat box, but the set of things that you can do is very vast and the ways that users might actually engage with the product is also very vast. And so a lot of the learning actually happens after you ship the product. And so product managers, you know, the generic PRD that you might write for an AI product is, Hey, like we're going to ship a chat feature. Yeah, it should be able to interact with the search bar and a few other things in Box. But actually, like a lot of the leverage and help that a product manager can offer happens after you ship that chatbot and is based on the things that you learn from users doing.

Yeah.

And, and so what we've seen is that product managers can look at what happens with your agent in production figure out the ways that it's misbehaving, and then try to articulate that in really good crystal clear English that an LLM can take and use to actually identify and understand problems.

I think what you're saying here, it was a, it was a big revelation for us when we started to do more and more of this, like, because for a while, like you were saying, like with our product teams and our engineers, we would say things like, we expect it to work like this. And you sort of describe that with all the different software development methodology requirements, documents and so on. But then at some point, like, you start to argue over what those words meant in that requirements and whether or not whatever the AI produced was actually producing that. And then we got to the point, like you're saying, where it's just like, we'll have the product manager write an example of the output. And that turned out to be not only, like, dramatically useful, but actually quite interesting and hard to do. It's different. It's different to describe, like, the kind of output that you want. Slightly easier than to say, when confronted with this kind of input, can you give this kind of output after doing a very long complex process? And so we have found exactly what you're saying is that like this methodology of the way that you build AI agents overall is different than before. Even with AI models, sort of single-shot style of like the older AI approach, the new way, but if you're going to give an agent a lot of leeway on how it handles a problem, then you have to kind of describe what what does good look like? We typically relate it to like, not, but like the idea of like what you'd expect from a person. Can you talk for a moment about like sort of this idea about evaluating results of an AI output and how you might like consider it to be like the way you test a person? Are these sort of equivalent in your mind?

I think the people that are good at testing other people are often good at testing models, but I think models and people behave somewhat differently. In many ways, I wish people would behave more like models. They're very consistent. They're very patient. You can talk to them whenever you want and they're very responsive, unless there's an outage of some sort. I think many of the same skills apply. One of the things we frequently talk to, talk about with people is if I use an LLM to evaluate another LLM, is that really actually doing anything? And I think the answer is yes, because if you think about the process of maybe like writing an essay and grading an essay, it's actually much easier to grade an essay than it is to write one. You can define, like, I think, you know, the SAT has a very clear rubric about what success looks like, but you can define what good looks like and articulate that in a way that's consistent and clear and measure it. And I think the same thing really applies with LLM evals.

Yeah, I think this, that concept is really interesting, which is like, it's different if I asked you a question like, what year was the Magna Carta signed, then like, like maybe, you know, and maybe you don't, compared to if I said, what year was the Magna Carta signed? Was it, you know, 1215 or was it like, you know, at some point you'd be like, no, that answer is right or wrong. I know it more than I did when you first had the question. And I think this kind of, we've seen this in our eval sets too, is that an AI model is smarter when it is validating than it is when it is trying to come up with the answer in the first place.

And I think there's also some quality guarantees that are structural in nature. So if you ask me the Magna Carta question, I tell you a specific year and the only thing I say is the year. Yeah, I think that is less likely to be correct than if I tell you the year and where I found that information or some additional background context about it. And I think you don't actually need to know the answer to be able to assess whether the answer contains a reference to a trustworthy source. Yeah.

So you mentioned something a moment ago, which I think is on the minds of a lot of people, which is there's this inherent non-determinism, not only in the way the models work, but a compounding effect of it with AI agents in addition to the output format. Like oftentimes it comes in the form of this natural language, which is sort of hard to use. And so I think people have noted that if you run AI multiple times, you'll get slightly different answers, in some cases very different answers. And then when you think about using that to solve enterprise-class problems, a lot of people would say, I can't do that if there's any non-determinism to the point of maybe it like, uh, interrupts their projects, their efforts, like just in general to try this out. They're like, until it can be 100% accurate, I can't use it. Um, can you talk about like your view on that, this idea that non-determinism is part of AI and how you would use it, especially when you're thinking about like enterprise-class kind of use cases?

Yeah, I think there's really two things you should think about there. So the first, when an AI system is very non-deterministic, it usually means that it is uncertain. So at this point, if you go to any model that released, let's say in the last year, that has at least 8 billion parameters and you say, what is 1 1? Yeah, 100% of the time you will get the answer 2. It might say 1 1 2, It might say the number 2, it might say T-W-O, but it is not going to hallucinate like 3, for example. If you rewind to maybe 10 years ago, you would get different answers every once in a while. And so I actually do think there's kind of this class of questions or type of problem for which the model is effectively deterministic. And you could argue what that means, like maybe 0.1% chance or something that it fails, but it is like effectively deterministic. And you can actually find this out pretty easily by doing evals. A good way to do it is to take a bunch of questions, run them through a model several times, and then if the answer changes, the question itself might be too challenging for a model to be consistently reliable on. And then the second thing for enterprise-class problems is I think problem framing and trying to think about how you can create the right validation or user experience expectation for a user is super important. So let's say you're working in a financial setting and you're building an internal tool for your employees to ask questions about your customers, let's say financial data. If the LLM is tasked with running SQL queries and looking at financial reports and then telling you the answer to the question, I think you should be very cautious. Like you should not trust those answers and send them directly. And I think it's actually a very dangerous thing in general, and a lot of employees are already doing this with ChatGPT. And so you have to be very careful. On the other hand, if you're working on something like a coding problem and you have, you're trying to migrate from Fortran to Java or something like that, and you have a bunch of unit tests for the system that you have in mind, then you can task an agent with trying to do this. And, you know, it may never actually be able to perform the migration. But if it does make some forward progress, you have a bunch of unit tests that you can use as kind of like a verifiability signal. And there I think nondeterminism shifts from being about whether you're getting the right answer or not to whether you actually or eventually solve the problem in the first place. And that's simply about how you frame, you know, the solution to the problem.

Let's explore the thing that you mentioned. So let's say that you have, you're doing financial analysis, right? So at some point you mentioned like you should be careful if you ask the model to do too many things because it can sometimes lead you astray either due to nondeterminism or other challenges. So let's break it down. So like, let's say that you ask a model to say, here's a bunch of documents, is this a good investment or not? Okay, so how does that rate on your scale of like absolutely will totally work or be careful with that kind of thing?

Well, I think it's probably highly context dependent, but I think one way of thinking about it is there's a range of ways that you could explore using AI to make this easier. So one thing you could do, which I've heard like is the case in a lot of medical settings, like for example with radiologists, is you could not provide any AI suggestion. So it's the human's job to figure out whether it's a good investment or not. And then when the human writes up whether it's a good investment and why, an AI system checks the human's work and says, okay, maybe that's good, but did you consider X? And so there the AI system is just providing critique and it's kind of a safety net.

And I think you're highlighting also the idea that like the context you give it is important. Like in this example, if I said, is this a risky investment or not? Like that is a very subjective question. Risky in what timeframes? Risky in what way? Risky according to sort of what comparisons? Like the word risky isn't necessarily something that's well-defined.

Right.

And so this kind of thing, you like a model will be more likely to be random in this case.

Exactly. Exactly.

So to talk about like, like, you know, some people want to do some more advanced things. And so like, how would you approach this? You know, again, with the power of thinking about tools, procedures, processes that help you get some of the benefit if you're giving it not just a question of like, here's a complex document, let's get the revenue out of it, which like something I think AI models can do fairly reliably, but then also like more complex instructions to do more complex tasks.

Yeah, I think I'll provide two tips. So the first is kind of a practical tip. If you're someone who's trying to do this today, one thing that works really well is actually asking a model to perform an analysis multiple ways and check whether the work is consistent. Yeah, we recently ran an eval study because there's a lot of interest right now in trying to use file systems and Bash as a mechanism for agents to work. And the basic idea is Claude code is really good and it's really good at using Bash. If we make problems fit that framework, will they work really well? We actually found that the answer is probably not, meaning SQL agents. Claude in particular is actually really good at running SQL and in most cases SQL is more than efficient enough. But we did find one useful thing, which is that if you give the model both SQL and Bash. It will run SQL queries and then it will also cross-reference the information it would run on the analysis it would do on the file system and make sure they're consistent. And if they're inconsistent, it would go and fix it. So doing this multiple path thing can be really good. And then the other tip I'd give is kind of like a managerial tip. So let's say that you have a bunch of people on your team who are really excited about using AI and you're trying to figure out how to avoid getting slop out of the work that they do. I think the key is to make sure that the human is accountable for the final response. And I think one tip that we've adopted internally at Braintrust and has worked quite well is if you use an AI tool, you can do whatever you want, but you're not allowed to share the direct output of the AI tool with your colleagues. So any synthesis or any writing that you share with other people has to be handwritten by you. And I think allowing containing the slop to be an individual kind of intermediate artifact. Yeah, it actually forces some discipline in how we work with each other. Now, this may change. Like, I think models for more advanced tasks are likely going to become so good that you should— shouldn't be adopting this procedure as literally as I described.

Yeah.

but I think like today it is a, it is a pretty good practice.

One of the sort of background themes that you're talking about is this, is this idea that like AI agents, AI models, they're going to do more and more over time. And you can kind of see it in sort of some of the benchmarks and some of the sort of the eval sets of people who build these things. But I think there's also a view where it's like some people are saying that, you know, a lot of AI projects fail, like the things that people think will work don't work. And when they try them, maybe they get it working, like you said, in a demo. And then when they go to try it in real life, it starts to fail. There's sort of the methodology that people who build agents in AI use, lots of eval sets, lots of ongoing experiments, like you mentioned. But then if you translate that to, let's say, people who are using these agents that somebody else built, or using platforms, or building their own agents in different ways, but sort of at a higher level, how do some of these concepts of evals and of these tools and this process, how do you think they apply? Like, because I think I met skeptics who say this doesn't work. I don't care how smart that benchmark says this is not working when I try it in my environment.

Well, first, I agree with those skeptics. I think that there are two kinds of evals, marketing evals and internal evals, and it's super important not to conflate the two. So a marketing eval, every time a new model comes out, the person who created the model will publish some sort of eval and claim its performance on a particular task. And the only purpose of that eval is marketing. There are people who work at these organizations who are micro-optimizing the performance of their model specifically on these benchmarks. And that's just reality. Like in database land, there's TPCC. And I think if there are any database nerds listening to this, you may resonate with this, but until it was very recently banned as part of the TPCC spec, the popular databases were doing specific TPCC optimizations in the database where they would recognize that TPCC was running and then behave a certain way.

They're very good at the test, not very good about the test measures.

Yes, exactly. And so the same is true with AI. And I think it is extremely good to be skeptical of marketing evals.

How can this be true that somebody says, wait, this model did so well on this really complex thing. Yet somehow can't work in my environment when I ask it something I consider to be way less complicated than, than these environments. Like, how would you explain that to somebody? And so to your point, like maybe they built a little bit towards the test, but some of those things are quite hard. Like what's the explanation for an enterprise person who's just frustrated with the idea that like maybe it's not working the way they want it to?

There's some level of showmanship that goes into making a model perform well on an eval, and some of that is For example, like how long the model can work on the task. Yeah. Sometimes models are given like days of GPU time to complete one task. Yeah. And that's not shown as one of the axes in the benchmark. But if you exclude the showmanship, I think it actually usually comes down to context, which is do the systems that you have deployed in the enterprise and the person who happened to engineer the AI system around the model that you're using, have they funneled the information to the model that is required for the model to be able to perform the task in the first place? Yeah. And a very good simple test for this is to ask, given the prompt that you show a model, yeah, could a human who doesn't have any domain context, like if you pulled a human who doesn't work at your enterprise company but is relatively smart, could they actually perform the task that you want them to?

Yeah.

And you'd be very surprised by how often the answer is no. Yeah. And if the answer is no, it's highly unlikely that a model will be able to do it.

And, and yeah, we, we see this a lot too, especially as you give it complex things, which is that multiple experts who are, who could evaluate it would actually disagree on the answer.

Exactly.

We, we had a, a fun thing we did recently with, so we launched an AI classification capability for like a security agent who can say this, this document looks top secret or not. And then a lot of people are like, what's the accuracy? But it turns out that like for many companies, the people who are in charge of it will disagree on whether that's a top secret document or just an internal document or others. And so it's interesting to see how AI in some cases is actually sort of exposing these questions that are maybe you didn't exactly measure in all cases.

And that ties back to the PRD point, right? Every time that happens, it's an opportunity for a product manager to be more clear about what you're actually looking for. And I think express that in eval.

And that was the answer we found was we should rewrite the rules on what is what and then have them agree on the rules. And then they actually start to agree with not only themselves, but also with the agents who are doing this. But you also mentioned the word internal evals as the sort of one of the keys to solving this problem. Can you talk a little bit more about that?

Yeah, internal evals are kind of like the spiritual successor to unit tests in the context of AI. So Internal evals are benchmarks that you have that try to measure how well a model or an agent or even like a piece of SaaS software that you procure performs on a task that you're trying to solve for. I think you guys do some really cool internal evals for the software that you're building. I mean, I think that building really great evals is probably the most important thing that you can invest in while building an AI product. And you mentioned something really interesting, which is Hey, a bunch of my internal evals might be failing, or I tried using something and it didn't work. Yeah, I actually think you should embrace that. So having a bunch of bad evals that are failing is a really good thing. Yeah, because it means that you have discovered something that is not possible today and you have framed it and captured it kind of like lightning in a bottle in a way that as soon as it does become possible, you will be able to automatically find out that that is true and then take advantage of it.

I mean, and to your point, like, let's say that like I go back in time, I can bring one thing with me, right? Like, and maybe I could bring my current agents and all the code and all the optimization we put into those and the techniques and so on. Or I could bring back my eval set. I think I'd almost certainly take my eval set because it just encapsulated so much like work from many people to say, this is what we expect in this very specific situation. And we can always recreate the technologies and then, you know, get models in. There's probably multiple ways to, do well on these in terms of what type of agent, what type of model, what type of methodology we use. But it's so hard to encapsulate what success means. And then we basically build that into the evals, which basically is the source of truth for us as to whether or not these agents are working.

Absolutely. One thing I think a lot about is how the implementation of software is becoming kind of disposable. So every time a new model comes out, you should be prepared to rewrite your entire agent, maybe you don't need the agent. So many things can change. But actually the eval is quite durable as things change.

Yeah, I think this point you made is something that you start to learn as you do, as you use AI recently. And it's hard for a lot of people. It's like, you're saying all the work that I did 6 months ago needs to be thrown away and start over? And you're like, yeah, that needs to happen. Okay, so you talked about the idea of these interesting tests. So the first being like, you know, give the problem to a human, the same inputs you give to an agent, give it to a human and see if they can, they can do it. And that's useful. The second is sort of like, what about if, do multiple humans agree on the answer? And that's sort of part of the establishing the eval. Can you talk a little bit about the breaking point or the idea of making sure that you get an answer where you see what happens when you give it something that you expect it to not work well? So for us, we do go out of our way to be like, here's some evals where we expect the agent to not answer, like not hallucinate, to say it hasn't been given the right info, to sort of respond a certain way. Like we found that this is a very important part of our evals because if you don't do this, then usually the thing that will start to fail is when people give it weird input and then it gives you what you might consider weird output.

Yeah, I think people, when they think about evals, they, in my opinion, over-rotate on this idea of a golden dataset. Yeah. And the golden dataset is this concept that you can commission, you know, it's almost like commissioning a work of art. Yeah, you can commission a group of people to create a North Star reference for a problem and it has inputs and it has perfect expected outputs. And if the AI system generates exactly the output, then you're good. Yeah. And there are several problems with this concept. The first is something that you mentioned, which is the model might produce something that is wrong and it looks right or it looks wrong, but it's not exactly the same as what the golden output is. Yeah. And of course you might not even have golden outputs. They're very, very expensive to acquire relative to the cost of acquiring a golden input. And so being able to measure outputs with signals like clarity or I would imagine in the context of Box, like citation quality. These are things that don't actually require reference output, but they're very, very valuable. Yeah. Another benefit of these kinds of measurements is that you can run them offline when you're doing evals in the development process, but you can also run them online, which means that if your agent generates something in production and it has a bad style or it doesn't include a citation or something like that, you can actually alert on that happening in production as well. Another problem with the kind of golden dataset concept is that in AI it's very hard to anticipate what users' input is. And so trying to anticipate that and put that into a golden dataset and then forget about it. Yeah. Or no longer try to enumerate new inputs is kind of a failed premise. What you really want is a systematic way of reconciling what you're testing offline with what users are doing online. And I agree, I think it's somewhat demoralizing sometimes to be a software engineer and not know what your job is nowadays or what you should be building. And the thing that you're building is no longer relevant, you know, a few months later. But I do think that it is a very fun technical problem and it's a very rewarding outcome to be able to build kind of like this reconciliation engine so that you can take what's happening in the real world and then efficiently find the interesting cases and test them and reproduce them locally.

Yeah, I think we've talked a little bit about like not only the idea of like developers who are building agents, like what we do at Box or what other companies do when they're sort of like building technology platforms or products for other people. But then can we talk about how some of this advice would apply to somebody who's using these agents, who's using something that they built at a kind of a on top of another platform where maybe you don't have access to the underlying sort of tools and processes to like, you know, like create a full dataset and run it. Like, you know, if you're thinking about some of these like, like enterprise platforms, like, you know, Salesforce or Box or ServiceNow or like Databricks, this kind of style of like these are— there's a bunch of different agents that you want to use, but then you can't necessarily feed them directly into a tool. That would give you these kind of eval outputs. And like you said, it's kind of expensive to generate these giant golden datasets. So like what kind of sort of points or principles of evals that, that sort of the developers have learned would be useful for somebody who has kind of a higher level use case in like IT trying to like, uh, use these agents to, to kind of, uh, try to accomplish something for their business.

Yeah. So the, the first thing I'd say is you should try to reuse all of the same principles, even if you're not writing code or doing it exactly the same way. And I think one really simple one is to try to test these tools out with, you know, like using their APIs or as users do a little bit of like a structured test, capture some feedback, maybe write down some inputs that you're testing on or some scenarios that you're testing and then try to quantify the results. And, you know, even doing something as simple as, let's say you're using an enterprise platform that you can only use the UI for writing down like 10 scenarios that you're testing and then writing down the outputs and grading them. Yeah, I think it's really useful because of the second point, which is that many of these tools are maybe in progress. Maybe some of them work for your business, maybe some of them don't. You actually want to be able to capture what works and what doesn't work so that in the future you can reevaluate with exactly the same criteria and you've built the same, as you said, like if you could rewind in time and bring the evals and you actually can create that same system for measurement, you know, even with opaque software. In many ways, AI software is opaque because it's not like any of us are reading the, you know, the weights of the LLMs that we're using. Yeah. And then I think the third thing is we work with a lot of customers on this and we also work with a lot of vendors to do this well. but I think you should try to push for transparency to some extent with the vendors that you work with. And you should also try to exchange these evals with the vendors that you work with because I think, and you can probably speak to this firsthand, they're really excited to actually get this data from you.

Yeah.

And it becomes a very direct way for them to improve the quality of their product.

Yeah. I kind of think in the future that like every tool that every, every platform that provides the ability to use AI and AI in particular, especially on more complex tasks, needs to provide some sort of way to either integrate with the eval system or provide its own eval system. Because otherwise, like, I think people fall in this trap of it works and then they don't explore the things you mentioned. They don't try it many times. They don't try it with a little bit different output. They don't see where it starts to fail, in which case that just means that the person who experiences a problem would be a customer or somebody who is early on like trying to see, oh, this does this POC really work? And then that's usually, I think, where a lot of disappointment comes in. You're like, I was so excited. And then I try something else and it fails. So now I'm not excited anymore. Instead of it being part of the development process where you're like, I am excited, let's make sure that we explore the other piece of this and then get to the point where we push it forward.

Yeah, I think nondeterminism is a pervasive thing. So every layer all the way to the consumer who's ultimately benefiting from the software that, you know, we're all working on, I think has to face it.

Yeah, absolutely. So last question. So let's say that somebody in the audience is watching. They are working on agent creation. They're working on in an enterprise trying to get agents to do more. What advice would you have for them to make sure that they're able to sort of use some of these concepts we talked about and then be able to sort of make sure the next time they try some agentic capabilities that they're less likely to experience a failure?

I think it's super useful for everyone to experience the workflow that we talked about, even if the tool that they're trying to test doesn't enable them to do that. And I think practically speaking, you have two options. You can either build your own agent and vibe coding tools make that relatively approachable today, or you can use something that is extensible enough. Like let's say you're using Claude Code and you're playing with it, even if you're doing some automation task instead of coding. You can actually take Claude code and point it at Braintrust, and then it will capture all the traces and stuff for you. But I, I think, um, it's super important to actually experience that workflow in a tool like Braintrust and run some evals and, and try some stuff out and see what's possible. And then I think once you have that experience, you can map a limited part of that workflow or, or somehow figure out your, a way to do the full workflow. Um, with any tool that you're testing. Awesome.

Great advice. Ankur, thanks for joining us and looking forward to chatting with you in the future.

Thanks for having me.

Thanks so much for tuning in. Uh, hope you enjoyed this episode and we'll see you next time.