Braintrust's Ankur Goyal on Why Evals Are the Core of AI Development

Today's conversation is with Ankur Goyal, the founder and CEO of Braintrust. Ankur is a repeat founder who started working in AI long before the debut of ChatGPT. He previously founded Empyra, which was acquired by Sigma, where he went on to lead the AI team. Now with Braintrust, he's helping developers build AI that actually works. In our discussion, we cover why evals are core to AI development, why taste matters in product development, what a culture of customer obsession looks like, and his approach to scaling and managing a team. Let's get into the conversation. Ankur, do you remember when we first met?

I do.

I'll tell you my recollection of it, which is I was, I think, a month or two into venture capital, and I walk into this conference room in our old San Francisco office, and I was meeting with you, and you were like, guys, I just got off of an international flight, like I think it was a red-eye or something, and I spent the whole flight building this product demo I'm about to show you, which is the kernel of my company Empira.

Yeah.

And I remember walking out and being like, I know nothing about venture capital, but that seems like the kind of person we should be in business with.

Well, it all worked out. Yeah.

Well, I want to mainly talk about Braintrust, but before we jump into Braintrust, tell us a little bit about yourself. How did we get here?

Well, my parents are doctors. Yeah. And I wanted to be a doctor my whole life until I took bio and I realized that I really don't like just memorizing stuff. And I took linear algebra at the University of Pittsburgh at the same time.

Okay.

And it like completely exploded my brain. Yeah. And it was one of those moments, like I started writing code for the class. It was one of those moments where I just saw like the rest of my life in front of me and I like, oh my God, I want to study computer science. So I studied computer science, and then I had another one of those moments when I took operating systems. And this is like a famously challenging class at CMU. I think the number of people that survive the class after the first week is 40%. Okay. And then I think about 10% of the people that signed up make it to the end. Yeah. And about 10% of those people get A's. Yeah. It's a really hard class. Yeah. And they tell you, you should expect to spend 40 hours per week on the course material for this class. But I loved this class, like I loved everything about it. And then I was like, okay, I wanna work on systems. And as a good Indian kid, I then tried to do research and I was bored out of my mind. So I had maybe my third existential crisis and moved to San Francisco and met Nikita and Eric from MemSQL. And I dropped out of school and joined MemSQL. And that was really fun. I was employee number 2 and I was there for about 5 and a half years. And then I left to start Empyra. We did some stuff for a while and then Figma acquired us and I led the AI team there. And at Empyra, we were doing AI stuff in the prehistoric, pre-ChatGPT era, and we were working on making document extraction work really well. And that used to be a hard problem. Now it's like super easy. Like, yeah, anyone can solve this. Like, we can go to your laptop right now and just solve it in 30 seconds. But taught me a lot about why AI engineering is really hard. Yeah. And we had to build tools to help us do evals really well. Yeah. And that's where I realized that, like, if you're building an AI product, there's no point doing anything other than evals. Like, everything should revolve around your evals. And if you do that, then you have a shot at building something that actually works. Yeah. At Figma, we had exactly the same problem. So we built similar tooling and Elad and I were talking and he pointed out, Hey, you know, you built the same thing twice, maybe other people have this problem. And so we talked to a bunch of companies and people had the problem, and so we built a really simple prototype which people started using like pretty much immediately. Yeah. And as you know and as I know, that doesn't happen very often.

Yes.

You know, if you have a really bad product and people are using it so much that they're complaining about it constantly and asking for features, that usually is quite special. Yeah. So then, you know, we kind of realized like, wow, there might actually be something here. Yeah. And then you and I chatted. Yeah. And that's how Braintrust really got started.

Yeah. And I remember when we first chatted about Braintrust at the time, we, you know, we at Greylock, of course, felt like LLMs were going to be a thing. AI engineering is important. All of a sudden, AI engineering had switched from the data scientists in the corner to every software engineer becoming an AI engineer. And so we were thinking a lot about tooling and, you know, meeting with founders. And we, you know, there are a number of companies that got started around different components of tooling, including evals. But one of the things that struck me in our first conversation about Braintrust was the way you framed evals to me as the core of what product development means in AI. Can you kind of share your point of view on that?

Yeah, I think AI is very much about defining the what, not the how. Yeah. No one really knows how LLMs work. Yeah. Despite what I think actually at this point, most people would admit they don't know how LLMs work. And so they behave unpredictably. Yeah. And yet it's our job to build good products. And so you ask yourself, what can you control? Yeah, well, you can control what you expect the LLM to produce and you can use that information to help you build a product around an LLM that is more likely to produce the thing that you want. And so that craft of actually deciding what you want the LLM to behave or how you want it to behave is really like the fundamental craft of building evals. Yeah. I think once you embrace evals, then you realize that what the LLM does is merely a function of the evals that you're producing. And if it doesn't behave well, instead of thinking about, okay, let me like tweak this wording in the prompt to say I'm going to lose my job or whatever it is. Yeah. You realize that the highest leverage thing you can do is actually try to capture what's going wrong in an eval so that any change that you make, you know, is actually going to work and it's not going to break all the other stuff that you've worked towards. When you really embrace it, you realize that the work that you put into doing evals doesn't evaporate every time you try a new model or you change some wording or you change your use case a little bit. It's actually very durable, and I think that's why it is the sort of true engineering that goes into building AI products.

When I talk to new founders, I hear this like a little bit of analysis paralysis of like, oh, should I start with OpenAI? Should I start with Anthropic? Should I start with like an open source model? Where do I like get started? And it seems like an easy thing to not connect yourself to any underlying model, but start with a really good AI developer platform that can extend across models is, is Braintrust.

Yeah, I think the only thing that you can be certain of is that whatever model you pick today is super unlikely to be the model that you're going to use solely tomorrow.

Yeah.

And so it's— I would strongly advise picking a development strategy, ideally, of course, using Braintrust that gives you a lot of flexibility in which models you're using and why. Yeah. I think in general, analysis paralysis is a very challenging problem in AI, and I would suggest just picking something.

Yeah. You know, so I remember early in the Braintrust journey we were talking and you said something like, I'm paraphrasing, But you said, like, I worked on Empira for many years, but now I think I understand what product market fit means. And one of the things that stands out to me about Braintrust is just the quality of customers that you're working with, right? Companies like Stripe and Instacart and Airtable and some of the highest taste organizations. Like, how's that all come to be? How are all these, like, really high taste companies using you?

Well, I think on product market fit, At Impura, and I think most startups, it feels like you're not going to be successful unless you accomplish something, and it always feels like there's some unless, like unless we get this many visitors to our website, we are not going to meet this aggressive goal that we have, or unless we ship this feature, we're not going to meet anything. And I think what I felt early on at Braintrust, I certainly saw at Figma, is you were always successful despite all of these things not existing. And that was a big mental shift for me. Like, it's quite interesting to see when people will just default use your product, and there's all these problems that you can keep fixing and improving, and unless you fix them, they won't keep using your product. But it's just a different place to be. In terms of taste, I think honestly the thing that we did well is that we were very specific about which customers we wanted to make a bet on. And make them really happy. I think that if you try to make everyone happy, it's really hard. But if you try to make a very small number of people that have similar attributes, like the companies that you mentioned, happy, then you can really zero in on what the problems are that they care about. And so we— it's very much part of our DNA that when we work with customers who we represent— we believe represent the future, we just roll up our sleeves and go all in. You know, the Stripe folks were at our office yesterday, and we had another really fun open brainstorm about the next 12 months of their roadmap and what they want Braintrust to help them with. And that, that's just kind of how we operate. And so I think that actually made it achievable for us to make some of these really great companies and customers really successful with our product.

One of the things I've heard when I talk to different AI companies out there who have looked at different platforms and chosen Braintrust and asked like, why'd you choose Braintrust? And they say like, the developer ergonomics and experience is really good. The product's built with quote unquote. What does that mean?

One of the mistakes I made at Empyra is that I did not create enough time for myself to think about product problems deeply. And I think this is a trap for a lot of founders because there are all these temptations all the time, like, oh, this VC wants to get coffee, let's catch up. Or this, you know, the corp dev person at AWS reached out and that's really exciting, you know, let me talk to them. But really, I think the only thing that matters is building a good product. And so creating the space to just talk to users and absorb their feedback and then think about it and iterate and solicit more feedback. As simple as it sounds, I think that that allows you to build a really good product.

So often I get asked by other founders, like, what does it mean to be customer obsessed or user obsessed? And you're the first person I think of, and I show people this photo. It's a photo of you in his, like, small home office on your laptop during this dinner party. Because I think a customer had complained about something and you were like, I'm gonna go fix this right now. And it seems like, starting with your own approach, but in the team you've recruited and the culture you've built, there's like a customer obsession that's hard to explain until you see it.

Yeah, I think a lot of it, at least on the engineering side, is giving engineers the permission to operate that way. One of the things I really don't like is sprint planning. I think the reason is that sprint planning associates your identity with a set of deliverables that you're going to get done over the next 2 weeks of time. Yeah. But what if a customer is using a feature that you built and they're having a really tough time? Yeah. Because you have a bug in the feature, which happens all the time. How does that— do I have to deprioritize the sprint that I committed to to help this customer? I think the answer is yes. Like, you should drop everything and help that customer immediately.

Yeah.

And unless you do that, you're going to end up with a crappy product because you won't solve all these paper cuts that accumulate over time. But that means that you lose roadmap predictability, you lose velocity in times like you might not ship anything for a month or two. Sometimes it's hard to keep track of individual engineers' progress, so it benefits you to have more senior folks on the team. But I just think you have to be really explicit about operating that way, and unless you are, the sort of default patterns that we've developed in the industry don't really allow you to do that. Yeah.

So speaking about driving amazing developer experience, I think one of the misnomers sometimes people have is to make these things possible, you actually have to solve really hard underlying technical problems. And I'm thinking about what you guys have built with Brainstore and what it enables in terms of user experience. Can you talk a little bit about that?

Yeah, this is like the third database system I've worked on, more years than I should probably admit on this recording, but One of the things that sets BrainStore apart that I think is quite cool is that we knew exactly what the problems were that we needed to solve. So for the first year of BrainTrust, we used the sort of standard medley of open source database stuff that everyone does in the space. And some of our customers who we have the pleasure of working with because we prioritize that group of people and are at the very edge of growth and product-market fit in the AI space overall, they started seeing exponential growth. And I think these were some of the first few companies to experience true exponential growth for their AI products. And our logging stuff just kind of fell apart. Yeah. Not only because of how much data they needed to ingest, but also because of the kinds of questions they wanted to ask from that data. Database systems, for example, aren't really good at ripping through a bunch of text, but prompts are getting bigger. People are sending more prompts. And searching for kind of the needle in a haystack of many, many prompts is actually really important. So we knew exactly the problem that we had to solve. And I kept having to hold myself back from building a system like BrainStore because I really wanted to. But it forced us to have an increasingly high level of clarity about what we needed to build.

Yeah.

The other thing that, that we benefited from is that the people who worked on it all knew each other really well. So the initial team other than myself was Manu, who's my brother, and Austin, who was the second engineer at Empyra, the last company. So we've all known each other for a really long time. And the three of us just sat down in a cave basically from October until January when we shipped it to the first customers and built the system. And it's very, very purpose-built for dealing with weird LLM-shaped data, like lots and lots of text, insane JSON that you could never dream of. It just works really well in Brainstore.

Often when people on the outside talk about Braintrust, I feel like the common analogy I hear is Braintrust is something like Datadog for AI workloads, which I think really understates and is too myopic of a view of what the role— of the role Braintrust plays for AI developers. Do you agree with that analogy? Do you disagree with that analogy? Like, what analogy would you draw?

I think there's a lot of truth to the analogy, and there's also it can also be quite myopic. So the myopic version of this analogy is thinking about both products as observability products. So if you think that Braintrust is like Datadog, but like the UI is a little bit better for LLM stuff, I think that's not the right way to think about it. And the reason is that you invest in Datadog as a business to achieve a particular outcome, which is uptime. And you invest in Braintrust as a business to achieve in a different outcome, which is quality. And I think that there's a lot of shared wisdom and insight that we draw inspiration from Datadog for to deliver the quality goal, but it's an inherently different goal. And I think it leads to a number of different product features. So for example, one of the most powerful features of Braintrust is the fact that you can take logs and connect them to datasets, and those datasets you use for your evals. And when you run evals, the code and prompts and stuff that you use for your evals are closely connected to GitHub and your codebase. We admire Datadog a lot. We partner closely with them. And yeah, I think there's a lot of inspiration to draw from what they've done for a different problem.

Yeah. So another kind of misnomer I hear in the background AI chatter is like, oh, well, there's gonna be a new company for agent evals, and there's gonna be a new company for voice evals, and these things. And so, like, what's your take on this?

Agents are a for loop with tools. Yeah. And most people that have agents that are working in production had roughly the same for loop working about a year ago. Yeah. And Braintrust was really built around, you know, those kinds of use cases to begin with. So I would just say, like, factually speaking, almost all of our customers are building agents on top of Braintrust. Yeah. And many of our customers are building voice agents on top of Braintrust. We are the first, for example, to build native integration with OpenAI's Realtime API, even before OpenAI had their own tools for it. But the deeper reason is that there are a lot of really hard infrastructure problems that go into supporting evals in the first place. Some of the problems that we solve are things like how to ingest megabytes per second of data that systems generate. And guess what? Agents just generate more data.

Right.

And so that's just a hard problem to solve. Another problem that we solve is running evals that might take a really, really long period of time, sometimes days, and accumulating the data in a way that's easy to visualize. And again, being able to run and orchestrate something that takes so long and capture all the data in one place, it's just like a hard infrastructure problem. And I think the fact that we've built really good solutions to these infrastructure problems gives us a lot of durability as kind of like the term in vogue evolves over time. Yeah.

So going back to the customers for a moment, I think one of the joys of working with you is like, I feel like every month, every quarter, we surprise ourselves with, you know, the velocity of customer adoption, you know, beating our revenue goals. Why do you think we're seeing such acceleration on the customer side?

Well, I think that customers are responding to inherent demand from users to use AI in their products. Those of us who use great AI products like ChatGPT, Cursor, etc., I think we start to expect that kind of experience from all the products that we use. And then I'd say secondarily, but still important, there are a lot of new startups that are forming that, you know, represent a fundamentally existential threat to a lot of businesses. There are a number of customers we're closing this quarter that are pushing really quickly to up-level how they build AI software because they're having serious discussions internally about how it's either figure out how to rebuild our product around AI or die. Yeah. And I think that upstream from Braintrust, these are really important drivers. Yeah. And then if you go downstream, you say, okay, how do I— given that I have 6 to 8 weeks to transform how I build software at this company, how do I ensure and maximize the probability that I build a product that doesn't suck? And I think if you look at the roster of customers that Braintrust has helped with that problem, it becomes a pretty obvious choice that Braintrust is probably something that, if you consider, will help you achieve that outcome. Now, of course, we have to prove it, and so we've also done a lot of work within Braintrust to hire an incredible go-to-market team, make it super easy for anyone to sign up for the product, try it, have really good docs. I'm happy to talk to anyone who wants to talk about eval. So we put a lot of effort into getting people up to speed and up and running with the product quickly.

One of the things you were sharing with me a few weeks ago, but there was a a customer where more non-engineers were using Braintrust than engineers. And like, I had this flashback to the early Figma days when sort of Figma tipped, and in many accounts, more non-designers started using the product than designers. Tell me more about that, like who's using Braintrust who's not an engineer and why?

Yeah, I think the simple answer to your question is that subject matter experts are incredibly valuable in fields like healthcare, legal, manufacturing, etc., where people are now leveraging AI for use cases that software wasn't really capable of solving before. I think some of the things we're seeing are just incredible, and you need people who are experts in their domain to be able to help solve those problems, and those are not software engineers. So again, fundamentally, the problem is— the problem that people solve is how do we get insight from people who are not software engineers into building a better product? Yeah. A simple example is if you're working in the healthcare field, Doctors are extremely good at contributing to prompts to keep models on track for, you know, very subtle and specific cases. Now, I think from Braintrust standpoint, if you're a company and you have non-technical people and you have software engineers and you're trying to figure out a common framework or language for them to operate with, it's a really hard problem. Yeah. And I actually think one of the most important problems we solve is giving engineering teams a platform that they can invest effort into, and the yield from that is a platform that they can collaborate with non-technical folks. And I think Figma is obviously a great example of that, and we draw inspiration from that as saying, like, okay, how do we enable our engineering stakeholders to easily set up brain trust so that non-engineers can use the product? One of the specific examples of that, we have this feature called span iframes, It allows engineers, if they want, to write custom code that renders a custom UI within Braintrust, and this is an incredibly powerful way for them to build a very, very, very specific experience just for their users.

So you shared a photo with me recently of the whole Braintrust team. We're growing fast.

Yeah.

What have you learned about scaling a team this quickly?

I think one of the most interesting insights I heard over the past few years was Jensen saying he doesn't do one-on-ones. Yeah. And that was a big unlock for me. I've been a manager for, again, more years than I'm willing to admit, and I'd never tried that before, but I tried it at Braintrust, and it's allowed us to be a lot more fluid and dynamic than if we had a very strict and complex hierarchy. We have basically no hierarchy at the company right now, and that's not to say we will forever be like that. And I actually think AI has helped a lot with that, and that's allowed us to grow really quickly.

I think on the team side, one of the things I'm impressed by is you don't just have a great engineering team, but you're very quickly building one of, if not the best, kind of scaling go-to-market teams in AI infrastructure. It feels like every other week I'm on a call with the sales rep from like a MongoDB or a Databricks who's coming over to Braintrust, which is awesome. How have you been able to do that, and what have you learned about scaling go-to-market?

The hardest lesson that I've learned over time is that you can't hire go-to-market people to solve problems that you haven't solved as a company. So I think one of the most common things that founders will do, I'm sure you've seen this, although I think you're too wise to encourage this, is like, we're not hitting our sales goals, we can't figure out how to sell larger deals, let's run a search and find a VP of sales who's gonna solve all these problems for us and help us find product-market fit. We didn't do that. You and I had dinner and you were like, "Hey, we're growing too fast. We need to bring people on, otherwise it's going to just fall— totally fall apart." And kind of like Brainstorm, by that point we had a very clear spec of who our buyer was, what they cared about, what are some examples of companies that have a similar sales motion, and sell to that buyer, and ideally are not competitive. And so we were able to almost handcraft the perfect sales leader and SE leader who know exactly what problems we need to solve. In both cases, they actually knew a bunch of our customers because they were selling products in the previous generation. That really helped us accelerate quickly. And I think when you have a product that's working, great customers, and a great team, it becomes a no-brainer for people who are interested in taking their careers to the next level to consider Braintrust as an opportunity. Now that said, I think we take recruiting as seriously as we take customer obsession. So we'll fly around everywhere in the US or the world, I guess, to meet candidates and take them out to dinner. Our customers get really involved in helping us recruit candidates. And I really like to think of Braintrust as like a community. If we have great sales folks working at the company, I think it's great for our customers. And of course, if we have great customers, it's great for the go-to-market folks. And so we just try to embrace that.

So yeah, there's customer obsession and there's talent obsession. You've got both. AI is exploding. Braintrust is exploding. Like, help us dream about the next few years for Braintrust. Like, what's ahead for this AI developer platform?

Interesting thing about AI is there's very little that we can predict. But if you look backwards, you know, over the past decade, despite all the things that have fundamentally changed, I think one thing that hasn't changed is that AI systems are getting simpler and more people are using AI. And those are kind of the core tenets that I think about. Braintrust is a really simple product. If you build simple agents and you build simple software, then I think you have a higher chance of success, and Braintrust really embraces that philosophy. And so I think we will continue to see simpler and simpler products that solve more and more interesting problems over the next several years. And I think that's paired with inherent demand for using these products. We're just scratching the surface on— if you consider some of the customers that we talked about, the number of queries per second that they solicit for their AI products versus their traditional products. And I think closing that gap and making every product experience an AI product experience is gonna be the, the journey. Uh, for us, I think that means continuing to innovate on the core platform. Um, you know, uh, supporting really, really large scale, um, is really important. Uh, and the other thing that, uh, we are really working hard on right now is actually simplifying our own product experience. So, As of the most recent vintage of models, we hit a threshold change or step function change where models are now sufficiently good at looking at their own work that they can self-improve. So we've seen incredible results with prompt, automatic prompt improvement, automatic dataset generation, digging through logs to find interesting cases, understanding what's not working by analyzing data from multiple experiments. And we already shipped Loop, which is the agent that's built into Braintrust. Well, we actually think it'll help us avoid building a bunch of features and maybe even remove some features of the product over time. So I think this is going to be another really big trend for Braintrust, you know, really becoming ourselves an AI-native experience.

Awesome, Ankur Goyal. Well, thanks for joining us today. And it's been just such fun to work with you, also to become close friends and like I think we're just getting started at Braintrust. So, so, so excited for the future.

The feeling is very mutual. It's been a blast.