The Uncommon Path | Ankur Goyal on Building Braintrust and Helping AI Teams “Study the Tape”

The ultimate value that we provide our users is a good user experience, and the stakes are really high now. And so you got to push yourself to figure out how you can really solve their problem the most efficient way. But we, you know, we want to help you be the Kobe Bryant of AI, give you the, you know, the film room that you can look at and understand why your product works.

You know, these are the important drivers of the business and love the grit and hustle.

Scene: Brain Trust, take 1.

Ankur, welcome to Uncommon Path.

Super excited to be here.

You know, I wanted to start off and ask a few questions, but you know, I think the foremost question is just the founding of Brain Trust. Like, when did this become so obvious to you? When did it just overwhelm you in terms of every thought and and when did you feel like, I need to build this, this company? What were the driving factors behind that?

Prior to Braintrust, I worked at Figma, and before that I started a company called Empyra, and we were kind of in like the stone ages of AI, pre-LLM. At both companies, we had this problem because we were building— at Empyra, we were building document extraction software, and at Figma, we were doing cool stuff in Figma's canvas using LLMs. Every time we changed something, we broke it. And once we sort of got a handle on that by building some tools, We didn't know where to get real user examples of people breaking our product, and so we had to figure out how to find those and then use them to test. And so we built this internal tooling, and to be honest, I didn't think too much of it at the time. I am pretty close with Elad, who invested early on into Braintrust, and we were just chatting about stuff, and he was like, "Why did you build this tool?" multiple times. To be honest, I didn't find it that interesting. In fact, I was pretty bored of the idea at that point because just purely from like a nerdy intellectual standpoint, I feel like it had been fleshed out. Like, okay, I know what the problems are. I know what UIs we should build. I know why it could be slow and like how to fix it and what the workflow should be like for people new to building AI products. I'm sure you're thinking about this and you're like, okay, well, that sounds like you should start that company, which is I think what Elad was thinking. But to me, I just felt kind of bored of the problem. But Elad convinced me to try building a prototype and see, you know, if people are interested in it. And what happened is I built like a really crappy version of the product in a week, and people started using it. And they didn't— they weren't just like playing with it. They were using it like every day and sending screenshots of the product and saying like, I wish this button opened this thing, or I wish I could ingest this type of data into Braintrust, or if I log an LLM call, I wish I could rerun it and see if I changed the prompt, what would have changed? Having worked on software for a while now, I think I know that that is a pretty unique thing when it happens. Like if people, even if your product is just terrible and our product was really bad at that point, if people are using it, I think there's something there. And that is when it sort of clicked for me that this was a really exciting opportunity.

It's an amazing story in terms of that pull and demand in terms of the market. When you look at the experience in terms of Empira and, you know, doing that in And you've kind of talked about that before. Like, when we built Iconiq, we built from first principles in terms of a lot of things we learned from other investment firms in terms of doing that. How did that affect the how in terms of how you built out Braintrust, especially when you started to see that traction in terms of users, that this is something they wanted, and you decided to build an exciting company around it?

One of the things that I've noticed, and I saw kind of the opposite of this at Figma, is Startups, especially when people start using their products, I think they get tricked into thinking that someone buying your product means that you have product market fit. And I think you have to be really careful about that. In fact, I learned, and I sort of saw the incredible amount of product market fit that Figma had, that you should almost be paranoid or skeptical that your product actually has product market fit. And so one of the things that we did is avoid kind of the common trope that you should start doing sales early on and actually get people to pay for your product. Instead, what we said is, hey, the thing is free, and if you use it enough that it is interesting to you, let us know. We didn't do any sales stuff, we didn't hire any salespeople. We just wanted to see, like, is the idea something that is useful enough for people to use? And what happened is, one, it kind of forced us to make a product that was genuinely useful. I remember early on I was trying to get some engineers at Zapier to get on a call with me to talk about the product, and they were just like, "No." They had no financial reason to do it, they were busy doing other stuff, and that forced me to distill what we were trying to do into a demo video that ultimately got them interested, for example. And what happened is people just organically started using the product in production, And then they said, well, hey, we're using this in production. We need to do some kind of commerce with an entity of some sort. Can we pay you so that there's a good legal framework around our collaboration? And that's how people actually started paying for and using the product in a commercial way. And I think being really explicit about not trying to convince people to use the product, not selling them the product so they feel like they have some skin in the game, it forced us to build something good early on.

It's amazing in terms of that skepticism that you have in terms of product market fit, but you can feel it now. We could hear it from customers, and the demand for Braintrust is exploding. Many companies talk about customer obsession as a value. It's sometimes on the walls. They look at Amazon as a, as a shining example. If I were to describe you as a founder, it would be customer obsessed. I think your LinkedIn profile says, you know, Chief Customer Officer or something like that in terms of Braintrust or, you know, customer support. You know, how does that show up for you? You know, what does customer obsession really mean, and how do you manifest that across a growing team?

The measure that I look at right now is actually when a user finds out that I am the founder of the company, are they surprised by that point? So someone, someone introduced us There's a really exciting company that we're starting to do a POC with right now, and someone who's high up there introduced me to the user. And lo and behold, I'd actually been talking to this person for a long time about some tracing stuff, and we fixed some bugs and shipped some improvements. And she was like, "Oh, I didn't really— I thought you were a support person." And I actually get really fired up by that. I think that, to me, indicates that I'm doing what I want everyone at the company to be doing, which is be in support and really understand and listen to customers and appreciate what they're asking for. What I personally find is that if I separate myself from raw user feedback and customer support tickets, then I lose empathy for the day-to-day experience of a user using our product. Pretty much everyone in our sort of leadership team is doing customer support. Even like people in marketing, for example, are working directly with our users on maybe issues that they have or ways that they want to collaborate with them. And so staying close to users is really important to us.

In the past, in terms of observability, it's never felt like the primitives in terms of the underlying architecture has moved as quickly as it's moving in terms of AI. I think one of the things that Braintrust has done exceptionally well, to your point around Lagora and others, is like working with the tastemakers that are doing some of the most interesting things in AI and having an exceptionally high market share around those companies. And now we're starting to see broader adoption in terms of more traditional companies and industries. You know, why is it so important in terms of the product development to be so aligned and so attuned to what's happening with the people at the cutting edge of AI?

I think AI is a very disruptive technology force. And when something is that disruptive, I think it can make a lot of people uncomfortable. And when people first started experimenting with LLMs, every user segment that we talked to had different preconceived notions about how they might take advantage of it. What will work? What will not work? We talked to a lot of people that had kind of a reductionist view of LLMs. They're very good at generating prose, but they're never going to be that good at writing code. They are not going to be as good as the models that we train in-house. And so let's experiment with them in the corner. And hey, if you help us experiment with them in the corner, maybe we'll play around with your product. So we talked to a lot of people like that. As someone who's building a product that hopefully has some longevity to it, you actually, you have to be very deliberate about which of those user groups you take a bet on. And just like they're going to potentially take a bet on your product, you have to take a bet on the customers that you work with and hope that the feedback that you get from them, which ultimately turns into your product, represents what the future is for how people are going to build on top of LLMs. And we made a pretty deliberate choice, which is let's look at companies that have really high-quality engineering teams. Let's look at companies that have preexisting product market fit. So they're not trying to use sort of vanity IP arguments or whatever around LLMs to justify their existence. Really good examples of that for our early customers are companies like Zapier, Notion, Stripe, Airtable, Instacart, folks that have incredible businesses and now are incorporating AI in really interesting ways in their product. And basically what we bet on is, okay, the way that these companies are going to build software around LLMs will become the predominant pattern.

And when you think about some of the other bets that you've made, like in terms of BrainStore, you chose to build a scalable underlying database given what you were seeing. And when we're on an exponential like we are, you know, the usage continues to explode and will grow, and so scalability is a really important feature, but it's exceptionally difficult to build a database into your product from scratch. What underpinned that decision? Why did you take the hard path relative to some of the choices that others seem to be making in the market?

I think it all actually comes down to making a bet on the right customers. So we, started working with companies like Notion, who were some of the first AI products to experience real product market fit. And so while everyone else was tracing, you know, a few hundred traces a day or something, Notion AI started working, right? And the number of traces that they were ingesting and also the types of insight that they wanted to extract from their data, it just started exploding. And so we were kind of at a point where we could have lost a bunch of business if we didn't take that problem really seriously and solve it. And, you know, I think others at this point have lost business to Braintrust because we are the only product that can actually operate at that level of scale. And so I think we saw the problem before anyone else did, just because we had the good fortune of working with the right customers. And then the other thing is that, you know, having worked in databases now through a few of these evolutions, like cloud was a really disruptive one in the past. I think when the conditions around the workload change sufficiently, it becomes very, very hard to use preexisting technology to solve new problems. And there are a few things that are interesting about AI The first is that the data is very, very large on a relative basis. So traditional observability data, the average span is about 1 kilobyte. In our world, the average span is over 100 kilobytes. Wow. In traditional observability, if you have a trace that's larger than 100 megabytes, the UI won't load. In our world, the average trace is over 150 megabytes. One, like the workload is quite different. The other thing that's interesting about AI is that there's a lot of PII and sensitive data that is very important to trace because those are the data points that you will ultimately use to run evals on and improve your product. In traditional observability, if you log PII, it happens, but it's a bug. And in AI, it is a core part of the use case. And so it's also very, very important for our customers to be able to run our software inside of their own cloud environments. And so if you take those two things and you try to cram them into existing database solutions, which we did like for literally every single one of them as we tried to solve this problem, it doesn't end well. And that's what led us to build BrainStore.

A better experience, and like this is the way that companies are changing and likely to operate in the future. One of the things that we've talked about is like Braintrust user experiences amazing in terms of that. And, you know, as we talked about, there's an explosion in terms of applications and AI and everything else, but there's still a shortage of truly exceptional products. How have you been able to prioritize user experience? Why is that such an important priority, especially in the markets that you're operating in, in terms of developer tools?

One of the interesting things about Vibe Coding is that it, um, it actually raises the stakes for for a product like ours to have a really good user experience. We can't, if our user experience isn't that great or it's not simple enough or customizable enough for a customer, then they'll just vibe code something, right? And I actually really appreciate that, 'cause I think it raises the stakes and pushes us to produce better products at the end of the day. And so I think that's one simple explanation for why it matters. Like if you don't have, a user experience that is sufficiently valuable to the end user, then it doesn't make sense for them to use it in the first place. And I'll give you a good example of this. One of our customers is Ramp. They have hundreds of AI projects that they're running across the company, and they were using us at first in a very data-centric way. So you could almost think of Braintrust as like a observability data warehouse for all of their AI projects. And we have human review functionality built into the product that's somewhat generic. So, you know, you have a trace, you can like label some stuff and improve it and you can configure a little bit about how it works. What we noticed from talking to RAMP is that they were building Streamlit applications for every use case that had PMs and subject matter experts looking at data, verifying that their thing was working and labeling it and so on. And we dug into that and we asked why. And the answer was that they could produce much simpler user experiences for those very specific use cases by vibe coding a Streamlit app. And so that's an example where we didn't meet the, the bar that you can meet by just vibe coding something. And so we spent a lot of time talking to them and thinking about it. And we realized that for this type of use case, trying to fight vibe coding is actually not a good idea. And instead, we built a mini vibe coding platform inside of Braintrust. So now, as a user, you can go into any project and say, I'm trying to create a labeling interface, and I want to just view the PDF, and I want to have like 2 buttons to the right of it. And if they push this button, you know, whatever this happens. And the nice thing about it is that at Ramp, for example, instead of having to vibe code something off on the side and then figure out how to securely and connect it to Braintrust and do all the data manipulations and stuff, they can produce a much, much simpler vibe-coded app directly in our product and achieve the same outcome. And so that's how I think about it. I think, you know, the, the The ultimate value that we provide our users is a good user experience, and the stakes are really high now. And so you gotta push yourself to figure out how you can, how you can really solve their problem the most efficient way.

It's amazing framing and way to think about it, and I think it speaks well to our strengths as a company. When you look at Braintrust, you know, we just crossed 100 people, as you said, you know, 2.5 years into it, and growing very rapidly, you know, how do you maintain the amazing culture and all the different traits that we talked about in terms of that, you know, customer obsession, that focus on user experience, you know, taking the pain away from everybody, that deep empathy. How does that continue to scale as you grow the team very rapidly?

You know, the strange thing, which I'm sure you've seen and experienced many times yourself, is that Even though we are a bigger company now than we were a year ago, it feels like a smaller company. Um, and the reason is that there's so much urgency and so many problems that we feel like we need to solve for our customers that we feel more understaffed right now than we did even a year ago. Um, and so I know that in number we are more people, but to be totally honest with you, uh, psychologically speaking, It actually feels like we're fewer people than we were before, and that is a very strange thing. I do think it points to the fact that we've done a good job of scaling our culture. And for us, the number one way that we do that is by avoiding internal meetings. We have almost no internal meetings except occasional all-hands meetings where sometimes investors come and hang out with the team and we talk about, you know, our last quarter and whatnot. And I think there are a couple benefits to that. To me, the most important benefit is that it creates time and space for people to engage directly with users rather than engage with each other. I've noticed that at a lot of companies, especially companies that are, that involve sales, the sales engineers become the customer and the engineering team listens to the sales engineers, and often the product managers listen to the sales engineers because they are much easier and more polite to get feedback from than the end users. So if you subtract the opportunity to have feedback flow that way, then the feedback actually comes from real end users. And I think avoiding internal meetings, like we actually don't have a, let's say, weekly or quarterly review where the sales engineering team presents their findings. Our internal protocol is that sales engineers should submit lots and lots of feedback, but the feedback has to be in the form of screenshots or quotes to Gong calls that specifically highlight the feedback from a user. And so I think that's helped us stay really, really connected to the end user and customer. And I personally think that If there's only one thing we can optimize for, it is the average employee experience at the company being very connected and aware of what problems actually matter to our end customer and end user.

You know, one of the things that I've experienced with you that I think is kind of speaks to my love language is like you are a consummate learner in terms of doing that. Like our mutual friend Satya Nadella, you know, is a learn-it-all and spends a ton of time with those at the, kind of center of innovation to really understand, you know, what's going on. But I've seen you do the same in terms of learning from your customers. But also, as you know, I've connected you with different people that we work with along our journey. You know, they've loved those experiences and you've just distilled down a lot of learnings from those experiences. You know, what's kind of framed that with you? Why is it so important to kind of be a student of history and learn from those that have come before you?

I love Talking to smart people, and I think people that have learned things through experience, including myself to a much lesser extent than the people that you've connected me to, I think they genuinely enjoy and feel a sense of fulfillment from being able to translate the things that they've learned over the years to what people like me who are discovering them are trying to do. And so You know, I think there's a wealth of knowledge that folks have that is just underutilized by entrepreneurs. And something that didn't come naturally to me early on, but I heard a lot of smart people talk about, was wanting to solicit reasons that you're wrong. And I've kind of tried to ingrain that. Like, when I talk to someone, For example, you connected me to Bob Muglia recently to talk about pricing. Yeah. I had a bunch of ideas and I just asked him to help me understand why these are bad ideas. And I think it was a really fun chat. Like, we went way over the allotted time and there's so many interesting insights he shared. But if you just think about it objectively, like, if you don't have a lot of experience in a particular area and you can talk to someone who really, really understands stuff and understand why your ideas might be wrong, you are objectively better off for having that information and that data. And I think it's something I really personally enjoy, and I think there's some kind of resonance with that, and people who've maybe learned something being able to pass along those learnings.

Such an important mindset in terms of doing that. I find that a lot of founders, particularly those that are seeing enormous success like Braintrust, you know, feel like they have to have it all figured out, but it's been amazing to see you kind of lean into these conversations, sometimes distill it down, and take the advice in terms of the way that we're building Braintrust or sometimes, you know, go in a very different direction given that you're closest to the problems. But it's just an unbelievable trait. We've talked a little bit about it in terms of the moment in time that we're at in AI, and we'll call it the Claude Code moment in terms of doing this. But as the products and tools become more autonomous, you know, what are the downstream implications that you see across the entire AI landscape beyond coding or beyond Claude Code?

I think there's this issue of comprehension debt. So you can ask an AI to solve something, and if you were to solve it yourself, of course, there's a lot of typing or thinking or whatever that you'd have to do. But a lot of the growth is from actually just comprehending the problems that you encounter along the way and learning about their solutions. And when an AI system one that can run tests and sort of debug and improve itself along the way, is solving stuff for you, it captures the comprehension, not you. And so when you're AI coding something, if you don't really understand the implications of the thing that you're building, when it inevitably breaks, it's very hard for you to take accountability for it. And I think this is where this whole slop thing comes into play. More skilled engineers who have accumulated comprehension knowledge over their careers, I think, end up prompting things that result in less slop because they bake in their intuitions and understandings sort of early into the process, but they still encounter this problem.

You've had a front row seat in terms of how some of the most ambitious teams are building in terms of AI. We talked a little bit about some of that earlier, but like What are the things that you see them doing totally differently that you think the rest of us will be doing, you know, months or years from now?

So I think the most ambitious teams are very, very dialed in to the product usage of the thing that they're building. And by that I mean the best teams are studying exactly how people use the AI features that they build. And trying to figure out how to recreate those problems when they're actually building and testing stuff. And this skill is a little bit different than traditional engineering. If you're engineering a system, it's quite easy to build a benchmark and simulate all the things that might go wrong. Yep. Or use an LLM to generate a bunch of tests. Or even if you're building a UI type of thing, play with it yourself and sort of be the Steve Jobs tastemaker type person. In AI, you don't really know what's gonna happen when someone uses your product, and so you have to adopt this mindset of being a student of usage. And I think it's no surprise that, like, really good product managers have become particularly valuable and impactful members of the product development lifecycle in AI because they are naturally wired to do this. We also see like really great product engineers step up to this challenge. And on the flip side, I think what I've seen is people who have machine learning backgrounds or data science backgrounds who really care about users, they are really thriving. And the ones that have historically not been as interested in how users are actually using the thing that they're building have had a little bit more trouble adjusting to this. So I encourage everyone that we work with to find people that are really passionate about the product experience and how end users are actually using the thing, ideally to be skeptical that it works, and to give them a tool like Braintrust so that they can really understand what people are doing with their product and make better product decisions.

You're a huge Warriors fan. Any, you know, takeaways in terms of dynasty era Warriors that have been, you know, inspirations to you in terms of building Braintrust?

Well, I was talking to Andre Iguodala. He said that the best NBA teams are the ones that take film room really seriously. And they are extremely passionate about watching how other teams play, watching how their team plays. When they make a mistake in a game, they immediately go to film and study it. And then he talked about how Kobe Bryant would watch film and then replay all the different things that could go wrong and all the different strategies that they might want to use that were informed from the film just meticulously all day. And he said brain trust is helping AI teams do that. And I thought that was just an incredible analogy for what we're doing. And I found it very inspirational. I think our whole team was really energized by it. But we want to help you be the Kobe Bryant of AI, give you the film room that you can look at and understand why your product works, why your product doesn't work, What are the things that warrant and demand your focus and then give you the ability to spend all day trying to iterate and improve and do a better job and then show up tomorrow with your product and hopefully do a little bit better?

That's an amazing story. I think the best investment firms also need to be cognizant of the, the game tape and how they played and cognizant of what else is going on in terms of the landscape, and it's amazing to have those sources of of inspiration in terms of Iguodala. I heard that he also helped us close a candidate as well.

You did.

So, you know, these are the important drivers of the business and love the grit and hustle. A little birdie told me that as we're announcing our Series B that you decided to handwrite notes to all of your customers thanking them for their trust that they put into, you know, our company and our team. Just, that's very uncommon. I don't think I've seen it before in terms of my career. What's gone behind that? I know everything that you guys do is very genuine. Continue and authentic? You know, what was the decisioning around that?

Well, aside from just how important our customers are, I don't think you would have invested if they hadn't taken a bet on us. And it's something, you know, that I can't understate the appreciation I personally and we as a company have for them. I think we're in a really unique moment in time where it can be difficult to know if you're communicating with a real person or not. Sometimes I receive messages or emails or whatever, and it's just very clear to me that an LLM wrote the communication and maybe even a specific LLM wrote that communication. And so we want to, we want to do something that's kind of refreshing and write something that is literally very— it can't get any more human than that. And just share our appreciation for people. And again, like the trust and bet they've taken on us in a way that hopefully resonates.

In terms of iconic, there are a lot of investors that would be thrilled to be partners with, you know, Braintrust. We got to know each other over time. I think we had a dinner with, with a friend, you know, Brett and others. And, you know, that led to a number of different conversations and some coffee at at Golden Goat, which is now my favorite coffee in San Francisco. You know, why did you choose to partner with Iconiq?

Well, I think first and foremost, I think you're awesome.

And the feeling is highly mutual. Thank you.

I really genuinely hope that Braintrust is the last company that I work on, and I want to work on it until, you know, I know I won't be too morbid about it, but until my, let's say, final years or final days. And so I, I want to do that with people that I enjoy their company, I have a shared set of values. When we talk, there is a consistent understanding of what is the truth and what matters and why. And if you actually think about that and you sort of subtract the razzle-dazzle of various people that you might collaborate with, there's a surprisingly small number of people that sort of meet that criteria. I think for any founder, it's quite easy to get caught up in, you know, people who are really wonderful, maybe having a certain set of accomplishments, but the way that they do things or the circumstances around those accomplishments being a little bit different than, than you. And so being somewhat honest with that has really narrowed the set of people that I want to work with down to a small number. The other thing, of course, is that at Iconiq you've had the opportunity to work with companies that have innovated in a similar space to what we've done. And I think there's a lot of really amazing learnings and network that's formed out of that that has helped us just accelerate our thinking on a number of important topics like the pricing one we just talked about. And then I think the third thing is just from a platform standpoint, the way that you approach connecting companies like us with customers and with candidates is fundamentally different than some of the other firms that are valuable and helpful in the way that they do it, but can be redundant at a certain point. So we, you know, having worked with Greylock and Andreessen and Elad on customer introductions and stuff, actually everyone has a different approach, and the way that they approach doing it has helped us get access to different customers. And the way that you approach getting access to customers is just— it's different and has helped us, you know, literally get access to customers that we wouldn't otherwise be able to. And so actually just strategically tapping into the way that you engage with companies has been very helpful.

Ankur, thank you for joining us on this episode of Uncommon Path. Appreciate you taking the time and sharing a little bit more with our community in terms of how you're building Braintrust, and why we're so excited to partner with you.

Thanks for having me.