How Hex Builds AI For Data Scientists, with Barry McCardel

There's an interesting thing called Jevons' paradox from mid-1800s. And this economist, I think his name was William Jevons, noted a weird thing that felt like a paradox then. And it's not obviously a paradox today, but you kind of have to think back to that era where steam engines were getting more efficient. So these were powered by coal and per, you know, whatever, kilo of coal, they were outputting more watts because we were just getting better at making steam engines. But the net coal consumption in Britain was going up. And he like couldn't figure out why would this be? These engines are getting more efficient. And to us today, it's kind of obvious because it's like people were just using a lot more. But like, this is an example of what's called a rebound effect, which is basically the cheaper something gets to do, the more of it you get. And it turns out that we just found like a lot of ways to use watts. In fact, like there's kind of this infinite demand for wattage. And I think about that sometimes of like, what are the tasks that AI will just make like much cheaper to the extent that maybe we don't get less of it, but we get more. And so it's like, if you kind of think of like the work a data scientist or a data analyst might do gets lesser, do you need less of them? Well, I don't know. That kind of presupposes that there's this like supply-demand balance today that like the number of data people in an org is the number it is because, okay, there's this many data questions that come in from the stakeholders and each analyst can handle this many. And so we need 4 analysts and they'll, but it's not really like that.

It's like the questions just increase.

There's an infinite number of these questions and things data people can be doing. And it's like, I wonder some days it's like, do you actually wind up with more Tina people? Like, wouldn't that be kind of weird?

Hey Barry, I'm really excited to have you on. Uh, I think you're— the B-A-R-R, the B-A-R-R, the Bar-Y and the Barry, uh, which might have been my first message to you. Like, you are Barry and I am Bar-Y.

Well, it happens a couple times a year that someone will text either of us Trying to get to the other person.

Yeah, we should just make, uh, make like group texts with everyone and we'll cover the bases. Uh-huh. But actually, speaking of that message, I reached out to you cold on LinkedIn for the first time. I think it was 4 years ago.

Wow.

Because I looked it up, it was 4 years ago and it was cuz I saw a demo for a relatively, at least looking backwards, a relatively early Hex product. I don't remember where I saw it. Could it have been Product Hunt?

We didn't really launch on product. I don't know where you would have seen that before. 4 years ago, I think our site was still a kind of like stealthy.

Yeah. Well, I saw a demo and I was a data scientist at the time and I was like, this looks awesome. This looks better than the Jupyter Notebooks and the piecing together of tools that I'm doing now. So I need to talk to this person. And a lot has happened obviously since that first meeting and you've built this phenomenal company. So thanks for joining and excited to, to get into it.

I am very excited to be chatting with you. Yeah, I knew you, uh, before you were at DBT. We'd spent a bunch of time here at DBT and then I was so excited you were at Amplify. And, um, I'm excited to talk about whatever you want to talk about, whatever you think people will think is interesting.

So I wore my Hex sweatshirt because we were just talking about how Hex has the most elite swag in the game. So maybe going back to when you had a stealthy website, because we're going to talk a little bit about Hex Magic and some of the AI initiatives that you have today. But if we rewind before that. I'm curious how you went from the self-proclaimed Excel jockey to landing on an initial product and focus for Hex.

Yeah, I mean, I, I've, uh, basically been a builder and user of data tools my whole career. And Hex is just like a super selfish startup. It's just like the tool I always wish I had. Um, and in some ways it's become that. In some ways I, there's a lot more we want to do that, um, to get there, to get to the full thing we kind of had in mind. Uh, yeah, I, I, in undergrad I started getting into data stuff. Um, I was, I like, was this like in a, we assisted in like a research lab doing social network analysis, which was really cool. It's kind of before data science was like hip. And then, um, I didn't know what to do with that cuz data science wasn't like a thing yet. And, uh, I went into consulting, which is like what people who enjoy like data and numbers, uh, in, at Northwestern in the early 2010s.

But it was on the precipice of becoming a thing. I feel like 3, 4 years later.

It was. And, and I had I had a friend who was like going to Google after school and that was like, whoa, you're doing what? And I think like at, if you went to school like Stanford or Berkeley at that time, it was probably much more common, but in the Midwest and at Northwestern, um, you know, people went on to do a lot of amazing things, but tech wasn't really in the repertoire now. And it's interesting because I've talked to people who have graduated from my alma mater much more recently. Even my brother, who's 7 years younger than me, he also went to Northwestern and it was like everyone in his class wanted to be like an engineer or a product manager. I was like, wow, this is the new, like, It's the new banking, it's the new consulting. But at the time that wasn't really on my radar. And so I went into consulting and I was like the guy on all my projects who just really wanted to do data stuff. I really kind of fell in love with Excel and was building basically these data apps, um, at airlines or the firm I was at was really focused on airlines. Um, and got really super into that, was doing this really complex stuff, setting up like illicit access databases under my desk, uh, the client and, um, Uh, my friend who was at Palantir was like, hey, you should just come to Palantir. Like we do this kind of for real, uh, like really just focus on data. And, um, they were at the time trying to get the commercial business off the ground and, and sort of transition it from being a purely services thing to more productized. And I think I seemed useful for them because I had kind of like tried to build these data products in the services context. So I spent about 5 years there doing all sorts of really super interesting stuff across industries and different problems. And I met. I didn't know it at the time, but I met, um, the people, Caitlin and Glenn, who went on to be my co-founders. I met a lot of our team who are here now and got just exposure to like stuff at a really interesting time. Like the mid-2010s in data was the advent of big data. It was like big data and data science and deep learning. These were kind of like all the rage and it was almost this first wave. Um, and I don't, I didn't definitely did not appreciate being there, how much on the vanguard. We were of this stuff. I think there was things not like not coming from like a data engineering background per se. I got much more technical while being there. You were kind of forced to, but, um, not being from that background, it wasn't obvious to me how much some of the stuff we were doing with like using commodity file storage for putting Parquet files in the cloud and then running Spark, massive Spark clusters on top of it and all this stuff, uh, that now, like, you know, gosh, 8 years later is almost like finally reached mainstream. Like data lakes are finally becoming a thing, the lakehouse pattern, like we had all of that in at that time. And it was a really interesting time to be there. And then I left there, went to a healthcare startup in New York and we had, it was very interesting leaving the Palantir bubble and going to a startup that was you know, just sort of a company doing data stuff. Uh, we, the company was focused on trying to bring data and tech to improve the way clinical trials ran. Uh, and, you know, we had a modern data stack in the, in the sense of having a data warehouse and CPT and a BI tool. Um, but we still had this problem where like 80 to 90% of our meaningful data work wasn't just cranking out another dashboard. It was like these really intensive analyses. We were doing forecast models. Basically these data apps we were trying to build. And given my background, I was, I worked really closely with our data team on trying to use data to solve all these problems. And it was basically like, I remember being in a leadership team meeting where I was like the only person around the table who was like, could run a Jupyter Notebook. And we were running these different scenarios for studies and I was projecting on the screen like a Jupyter Notebook where I was like—

They're like, how do I even access this?

What is this? What is this?

Yeah.

And someone asked me like, is this the state of the art? And I was like, You're like, yeah, it is. I don't know. Like, and so it, I started this journey as a buyer, not a builder. And I, I think that's common for a lot of companies. I don't know, with good products is like, you, you gotta have a sort of pain point that you intimately understand. And, um, it wasn't until I went and talked to a bunch of other people working on data teams, uh, in New York, they all kind of said the same thing, which is like, yeah, we, we have this problem too. And if you find something to solve it, let us know. That I, it kind of clicked, like maybe we should do this. So yeah, that's the journey.

I wanna hear actually about that because, so I was a data scientist too. I've also been in the show, share screen with your Jupyter Notebook while everyone tries to follow along. Pretty hard to collaborate on.

The, the workflow everyone does is like take a screenshot of a chart.

I was gonna say I've screenshotted, put it in the slides, changed it, put it in a PDF of a deck that you send in an email. Yeah, 100%. So what, what was the process of talking to other companies look like? Because there are disparate tools that did exist and covered part of what Hex does today. So, you know, there's a big BI market and then I would say the Jupyter Notebook market, you can use a Jupyter Notebook for free. I've heard you talk about commitment engineering and what is that and how does it help you define the scope and figure out that that's actually what people want?

Yeah, it was interesting because at the time, a very common thing when you're starting out is You'll hear people be like, oh, well, like this is solved by this other thing. Or, and, and sorting as a founder, sorting through that and deciding like, am I getting good signal that this is a bad idea? Or am I just talking to a bunch of people who are used to the status quo and I need to like push through it? It was like very difficult to discern. And yeah, people were like, well, you know, Jupyter's free and there's JupyterHub and there's all these Jupyter extensions. And this problem you're describing is solved if you install these 3 extensions and you get all the versions right. Don't, don't upgrade that one to the newest version because it'll break these other ones. And if you get them all running on an AWS thing that you're setting up and you have someone managing it and like Mercury and Saturn are aligned in the sky, it'll all work. And it's like, it is just such a pain in the ass. And it like wasn't solved. It was like a, it's like people were describing how you could duct tape something together, not build a good product. And I, I do a lot of respect for people who have worked on the Jupyter project. It's super popular and certainly a source of a lot of inspiration for us. Um, but I think it's, it, it never, it was not clear that it was ever going to become the product that we, for all of its, for all of its charms and benefits was ever going to become the product that we, um, thought should exist. And so that's, um, you know, kind of how we got to the, the starting point. And, and a lot of this was grounded in talking to early, you know, I would say early customers, but they weren't customers. They were just like people we talked to who were like on data teams. And the commitment engineering philosophy is kind of like the way to find the wedge in when you're building a new product. It's a really tough moment, and this is true at the beginning of a company. It's also true, uh, even as you go through and you're building new products over time, which is no one wants to use your early thing because it sucks. Because early software sucks. It's just like, it is, it's fine. It, it's a reality. And I think a lot of founders go in trying to be like, Um, hi, do you wanna, let me pitch you my thing. Do you wanna buy it? And it's really tough the first few months or even years of a company before you've really hit PMF to like get into a loop where you're actually able to, to get people giving you the feedback you need.

And if you make it really good and purchasable, if they don't want it in the first place, you've spent too much time doing that.

Exactly. So commitment engineering is a way to break this trade-off and The, the idea is that you, you set up these give-get loops, these commitment loops with a customer or a prospect, a design partner. Um, and they don't have to be money. In fact, it's better if they're not money. Um, where you can go and say, hey, you know, I've been thinking about this problem. Do you have this problem? Yeah. Would you take a half hour and just talk to me about this problem? I'm thinking about doing something to solve it. If they say no, then the problem isn't big enough that they're willing to take 30 minutes of their day. Like that gives you some signal. If they say yes, you know, do the call and at the end be like, great. If I came back in a few weeks with a first prototype or some designs or whatever of something that could potentially solve this, would you take another 30 minutes and chat with me? And again, like you're kind of asking for a next commitment before you go and do a bunch of work on spec. You're kind of asking for that commit.

Unless they just really like you, Barry, and they want to hang out.

Well, that's, yeah, I think the people are polite and that, that is part of the reason you have to ask for a commitment. And, and I really push people to be like, If they say yes, be like, great, what time works? Let's get out our calendars. And you'll get people hemming and hawing, being like, well, you know, it's, it's review season. You know, we can't get anything done for 3 months and I can't, like, whatever. And, and when you're a founder, you want these, you have these happy years and you have to find ways to test people's commitment. And you can ride these commit loops all the way up through a really big business where you're kind of going to people and you're saying like, Great. Would you click, would you grab more time and click around in a first prototype? Or would you use this in anger? Would you demo it to your boss? Would you do another session with me where you invite people on your team? There, there's all sorts of things you can ask them, right? Like to, would you put real data in it? Like, and you can test. And then when you actually ask for money, that's like the la— you know, that it's the ultimate commit, but it should be obvious cuz you've kind of been spending time working with them on this. So. I think that's a really important thing and that's what we did early on. We had some early people, we didn't ask for money at first. We were just interested in them using it and we were just validating, like, are we doing something interesting enough that they're willing to spend more of their time or social capital or, you know, whatever it is. And, and that's way more important than money early on. Like the, the most important thing early on is not revenue. It's, it's really high quality A lot of feedback and customer partnership.

What did you identify through this series of commitment engineering as the thorniest solution that, that people wanted?

Well, we, we had a big decision to make early on where there's a, there was a way to potentially solve the problem that we were set out to solve that would've looked like try just kind of being a last mile that was like, okay, what if, what if actually like people, we kind of accept that people are gonna edit their things and Jupyter or other tools or whatever. And then maybe we're the place where you send stuff to share and we're like the collaboration hub. And you know, the first version of Hex actually kind of looked like that. And what we found was that wasn't valuable enough for people. And actually the thing they kept asking us for was to actually just do a better job on the editing experience and have it be that, that actual like editing experience be online and collaborative and great. And there was this moment of just sort of like following our nose a little better, following what people were asking us for. Um, and actually having some people like not be willing to commit to more time because the thing we were doing felt like it wasn't actually solving the problem deeply enough.

No, it's interesting though, because I can see that hypothesis. If you're like, where can we save time? They already know this workflow. They're already in Jupyter Notebook. You learn it at school. Yeah.

You know, why do we want to try to displace that? I'd had an experience, I'd had an experience early in my career building, uh, like working on a, uh, platform that we had this like IDE at Palantir. We were trying to get developers to switch IDEs. Like you only do that once, like try to force people to switch IDEs. It's like hard.

You have to learn it. You have to feel comfortable.

I think we came to believe that we could do a good job on it. And then we just started commitment engineering that. So we had like a really basic janky version and just kept asking like, what else would you want to see? Cool. Would you grab more time or would you, would you use this again? Cause you'd have some, sometimes people will be like, I can't really use this in the current state. Like, great. Would you give it another shot and set up time to give us some feedback if we did X?

Yeah. And I'm sure some people said, if you did X, I'll give you another shot. And you get, you get input that way too.

Right. Which is, which is tricky because, because there's a, you can take that at face value. You can also have people who are something, either they don't know what they want and they're like, well, you know, uh, yeah, you know, if you had this chart type, I might use it again. But like, is that really the full inventory of what it would be to get to PMF for them? Hard to say. It's like a give a mouse a cookie problem. Um, there's also like the, the sort of like they're trying to be polite and give you some feedback thing. And so it's just a really hard and confusing time. And I think the commit, asking for those commitments, and if they're like, great, uh, I will use it if you do this, be like, great. Will you demo it? Will you like demo it to your boss? Will you like, can we set up time right now? Will you intro me to like your other coworkers, like really get them to put skin in the game before you go and like spend weeks building a feature on spec.

So I think this is a good transition when I think about, you know, how do you have some conviction that this is the right product direction before you put resources and skin in the game? I think you did a really good job doing that with AI, like you were relatively early moving into AI world. And so maybe I would start with why did you hypothesize that data science workflows were a good place for LLMs in the first place? And what did you do about that?

Well, there were kind of two phases to it. There was April '22, so this was pre-ChatGPT. Um, I got access to the GPT-3 API because I'd seen some demos online, people doing interesting software-based GPT-3. And we had a hack week that April, and I made one of our engineers for hack week integrate with it. And I remember during the demos, everyone was like really hype on it, but, um, it wasn't like a thing people were obviously asking for. I was kind of like, ah, we should, you know, do this more. But we were a really small team. We had other priorities. And, um, in, in one sense, you could look back and be like, oh man, we should have seized on that moment because we could have been early to, even earlier to AI. On another, it's like, that probably wouldn't have been time well spent because just the tooling models and the infrastructure were so primitive at that point. Um, and we had other stuff to do, but yeah, so there was that phase and I was just following it super closely and chatting with other people. Um, another Amplify portfolio company, Runway, um, Chris over there, I remember seeing demos of what they were doing and it just, it just was kind of obvious there, there was a lot happening here and it was starting to really work in a way that when I had been exposed to sort of the first ML hype wave in the mid-2010s, it wasn't actually like a lot of stuff wasn't really working in a generalized way, or it was just very hard to get working. So, and then, and then after ChatGPT, we just, you know, a lot of people started paying a lot more attention to it. And, uh, it was February of, I guess, 2023, um, when we launched the first version of our magic features and, and Um, and so we were well primed and well positioned to sort of dive in on it. And in a funny story about the first, our magic features were the first version of them took one month for one engineer and one designer and me. It was pretty straightforward. It was, you type a prompt and we generate code. Um, it was almost like, you know, kind of what you would get in ChatGPT of going there and we didn't, weren't passing in a lot of extra context. We didn't have vector database that we were running or anything like that. It was pretty straightforward. Um, and then we just started investing more in it. And I, and I do think it's an interesting thing I think about even now, which is like, when is it actually, um, I think a lot of founders, a lot of companies, just large and small, are, are putting a lot of effort into AI stuff. And it's an interesting question of like, where can you put energy into that and get product out? And where, um, is it still effectively like a research field where you're gonna put energy in and like, it's unclear where you'll get the return. And I, I say that 'cause like, um, you know, I think we're a pretty, we're, we're a team that has been building like data software for most of our careers. Yeah. Like, like pre-Hex, you know, like this is kind of like my co-founders is like literally all they did. Their first job was at Palantir and like all they did was be engineers building data products. Right. So like, we, we, I think we like a good team that like can look at a, a product thing and be like, okay, that we can do that. That's gonna be about this much effort and we know how to go and do that. Um, I think the interesting thing with AI is like it confounds a lot of your intuitions and it's not clear what is going to work ex ante or why it's gonna work, or you do all this work to get it working with a current model and then a new one comes along. And I think that's just a very interesting thing. And I look back even at some of the, the time we've invested in the AI stuff, and I can point to stuff where I'm like, yeah, we did a great job. I'm so glad we did that when we did it. And other stuff where it's like, man, you know, we probably could have just like waited 4 months.

What's an example of something you think you could have waited 4 months?

We spent a bunch of time on, um, like edits, uh, like when you were, when you would ask our magic features to edit your code. In a cell, like I want to edit this Python code, um, it would generate, it would basically stream back, you know, we pass it the code and our prompts and then ask it for an edit and we'd stream back the edit. So if you had like a 40-line cell, which is not uncommon, you're waiting for all 40 lines to stream back and then you see that it like changed one line, right? So that's annoying. So it was like, okay, well, let's invest all this time in like, um, doing diff-based edits and really get that latency down where we're where we're just streaming about less back and we're doing like a really good engineering project. Our team did a good job on it. And then like a month later it was like 4 Turbo or 3.5 Turbo, whatever generation model was the time I forget, came out where it was like, oh, like just doing the old approach with this new model would've gotten us within like, you know, close enough. And it's like, oh yeah. And something we've just said over time is like, it's, um, like live in the future a little bit is okay. And, and you can just extrapolate some of these things and assume the models are gonna get better. And it's, and it's interesting cuz there are, you know, there's a lot of conversation in the like techosphere right now about if we're hitting some like plateau on LLM, um, scaling when there's certain things where we're like, okay, we're gonna assume that like this is just gonna get better and it's not worth optimizing now. There's other things where it's like, well, maybe this is kind of what we can expect.

What do you think is gonna get better?

Well, there's a question on reasoning that I'll set aside for a moment. Um, cause that, that probably is going to get better, but it's not clear how much better it's going to get in like successive base GPT or base Claude models. Like, you know, there's the 01 series now. So we'll set that aside. I think there's stuff around like latency and speed and, um, for what I would say, these more straightforward tasks. Um, and cost around that. So there's certain things that we're like, yeah, if we did inference for this every time a user, like, you know, basically typed something, that would be, you know, expensive and it would add up.

And yeah, but our assumption is that will just go down over time.

It's just going down. And, and like, I have a chart, um, in a blog post I published last year about why we're not charging extra for AI. Um, we can talk about that more and go deeper on that, but like, It's a chart that just shows the relative cost of inference over time from when GPT-3 first came out to like the, um, 3.5, the, you know, whatever the last version of 3.5 was. So it should update it even with like 4.0 Mini now. And it just is like a crush. It just comes way down. And it's like, you know, I, I think you can effectively extrapolate some of these tasks down to a, I wouldn't say down to zero, but like down to like really low.

So why don't you charge? I mean, I think that again, you were, we're still relatively early with HexMagic, and what we saw a lot of companies do is come out with an AI SKU. If you have these AI capabilities, pay up. And so was that a difficult decision? Was that a debate? Was it obvious to you?

Uh, it felt obvious to me. I, I'm also like, uh, you know, I don't know, ask your, uh, ask some of my board members and they'll tell you that they think that we leave too much money on the table sometimes, which is true. It's true. And this is another example of leaving money on the table that we could have gone and done a cash grab. Like, people love our magic features. We did an event last night, people were coming up and raving about them. I was like, we, we could charge for this, but it never felt right to charge for it for, for very principled reasons. And I'll tell you why. One is, um, the, the cost actually isn't that bad, that bad, and it's getting cheaper. So it's, it's actually like, for if you're doing it to like cover your margins, I actually like, I'm not, like, it's not clear to me that that's the right way to think about it. Two is having an AI add-on presupposes that there's a version of the product you'd want to sell without AI. And that's true today. That's true for all these products with AI add-ons today. But I don't think it's going to be true in a couple years. And I equate this to like, if you were charging extra for cloud in like 2014, 2015, like exactly 10 years ago, like that is a thing you could have charged extra for theoretically. Like now it's just like, like the next wave of SaaS started and it's just like, yeah, this is how this shit works. This is how software works in the future. And like, if you're starting a new company today, you're not going to charge extra for AI. You just assume AI exists. And for certain things that are incredibly like token consumptive, maybe you are, you have some like consumption-based thing on it, but you definitely aren't going to have the like little add-on thing that's like with the sparkle emoji that's like upgrade to the AI version and Cause it's just like how software is going to work. And I just felt that doing the work to charge extra for a SKU, you know, great. It may be a short-term cash grab, but is it like, it kind of, it almost like wires us internally even and like sends a signal to us and our customers that like, oh, this is a bolt-on thing versus like, we actually think this is pretty fundamental to how this stuff should work down the road. Um, and yeah, I, I just. I find the AI add-on thing to be a bit of a cash grab. And I've talked, I talked to my friends, founders of other companies that have done it. And I understand their reasons and rationale, but I, I don't, I don't think that will last very long. Like, I don't think that we will. I get bugged. I fucking hate it. I get bugged constantly because I'm like the admin of our Slack workspace to like upgrade us to the Slack AI. And it drives me nuts because I'm like, this should just be how Slack works. And I bet you they know it in their bones too, where they're like, yeah, do we really want to sell a version of Slack with like shittier search? Because that's what they're doing today. They're selling the, the base version of Slack as like worse search because they're not willing to, you know, because they're going to charge extra for the AI thing and make you pay for better search. You can get away with that for a while, but there are people building next-gen AI-first messaging products, and at some point they're going to need to just reconcile that. And, and I, I would much rather actually just have them be like, our C cost is going up by whatever, then get nagged about these add-ons that I'm just like, yeah, it feels dumb. So I don't know, maybe that's just like, I'm, I've got an aesthetic sensibility around it.

Um, I agree with you. And I, you know, I think that right now in the, we'll call it code gen space, but in this case, data science gen. AI has almost become, maybe in Silicon Valley circles, table stakes in code gen, certainly not everywhere. And you have these two camps, one that looks like you have the agents go and do some labor for you and return with some labor output. They're very imperfect today. And you have AI as a copilot helping you code. So I'm just curious philosophically how you think about that in data science workflows and, you know, what does it look like to have an agent with a more DAG-centric approach which I know you all have in Hex too.

Yeah. I mean, I think both will work. Um, I think that, uh, in data, uh, there's a lot of interesting things in our market. Um, and one of them is that there is a, you have this variance in basically the level of quality that you can come back with for your user and have it be acceptable. So, so CodeGen, like if you look at Copilot or Cursor or Um, whatever else is out there is a lot of them now. Um, the users for that are 100% software engineers who know how to look at something and understand if it's good or bad. Now that, that might be a slight exaggeration cuz I use Cursor and you would not call me a software engineer, but like you can look at the output and see this is ridiculous. I wrote all the code for it like pre-GPT and I can kind of look at it and I'm like, I do the lazy, just like. Brute force, like I keep hitting errors, just like keep having it try to fix them. And like, you know, I do that instead of what I would've done before is like go to Stack Overflow and that's fine for my workflow. But like for, for most people using it, it's like, oh, you like, my dad isn't just gonna pick up Cursor and start coding up a website, at least not today. Um, but the, the point I'm trying to make is like those, those products can be wrong and still be useful. And that's true for us when it's like, okay, we've got a data scientist or a data analyst and we can write them some SQL or Python and they can look at it and be like, Hey, that's a little wrong, but still useful. I'm so glad I had that. And you, in those, for those folks, you can have an agent that goes and does a whole chain of reasoning and comes back. And they can look at that and be like, that's wrong, but still useful in the same way they might for a junior person or an intern.

Although in the case of data, it may require, let me look at all the steps.

Yeah, that's right.

If it goes from SQL to analysis to plot, let me make sure these all check out.

Yeah, yeah. And you need to introspect that and explain what's going on, and there's a lot of like hard stuff in there. And you know, we're doing that, and, um, it's cool. Um, but, but then you have this much broader sort of population of people who interact with data in like a lower intensity way in the organization, who you can't be wrong.

Yeah.

And, and you can't be wrong and still useful.

They don't know if the output is wrong.

Yes. And that is one of Um, the things with is, so we were talking about AI add-ons, like one of the things with AI add-ons is like, you know, in the early days, one, another reason I didn't want to charge for it is because like these things were still kind of buggy and it's like, you know, you're going to charge for something that's basically a prototype. It's better now. But the, the other thing I think about is there's startups in a lot of categories now, but in data especially, it's this very pernicious problem where you have these startups that are building like explicitly like AI data tools. The problem there is you can't, you, the, if your PMF is predicated on anyone in your org can ask a question and get back a correct answer, you better damn well be right. Like 100% of the time, like maybe 99%. And if it's anything south of that, it's just a very brittle and very like, um, precarious thing. And I think, um, uh, we, we like where we're coming from because we have we can layer in these AI features in a way that are, um, accretive and beneficial. And, um, we can learn from and get, you know, better and better PMF and better, better understanding of how these things should work.

In other words, your end users know if something goes wrong. And I see this all the time as an investor.

Yes, our, our, our like core user base today, right?

Yeah, your core users, um, or they can check it. And, um, I see this a lot as an investor too, where I'm like, I don't think we're ready for autonomous data agents for marketers who've not used SQL before. I think it was in late October 2020, I had access to the GPT-3 API. Basically worked on this hackathon where we made like a crappy Jupyter Notebook that did text to SQL, just having fun. Anyways, we worked on this hackathon and it was basically an AI-powered notebook, but it really only worked if we used this very specific Kaggle dataset where you didn't have to do joins on the columns. So it was an awesome demo because it was very early to people seeing it.

And yeah, the other thing you find, by the way, is when you're using a public internet datasets, yeah, the model has seen those. Yeah, yeah, yeah. So, so like a funny thing we see, you see these, uh, shadows of it as like, um, like there's a, there's a, all of our conference rooms actually, funny enough, are named after like, uh, famous datasets. So we have a, we have a room called Penguins. I'm in the room called Iris right now. And the funny thing is that like, um, we found that our AI evals when we were first writing them were really, really accurate for some of these demo datasets and then not at all accurate for the others. It's like, why? It's like, because it's seen the Penguins dataset. It hasn't seen like Hex's whatever, like our internal data. And, uh, uh, yeah, so it's, it's, it's the, the demo, getting the demos working is like. One thing, getting them actually useful. It's also not clear to us where, um, the AI-first thing actually negates the need for everything else. So like thinking about other categories, right? Like if you think about, uh, design tools and Figma, like it's not at all clear that like an AI-first Figma is a thing because like if you want to edit 100% via prompts, maybe, but like most people are gonna at some point be like, well, I wanna move the rectangle like one pixel to the left and change that font. And like, yeah, like, I don't know. I, I like point and click for that stuff. Like, or I wanna be— and, and, and so there's almost this like multimodality in terms of interaction that you want. And I, and I definitely think it's true in data and it's not at all clear to me that, um, you don't need to do all these other things, um, to have a, a, a great product. Now I could be wrong.

Well, well, this is one of the things that even pre-AI I really like about using Hex. Like I can do some SQL and then I can, kind of play around in Python and I can move around between different languages or modalities, as you would call them, like, how can I then do the other thing in the tool where I'm most comfortable doing it? And it may not look like, let me try 80 other prompts until I potentially get it right. It may look like I just need to change this cell.

Yeah. Like sometimes you just want to change the chart color or just change the column name and just like do it.

Um, yeah. So this is, this is a benefit of having the constrained UI versus You know, I'm just writing a prompt into the ether and hoping that the right thing comes out of it.

Another, um, I didn't think about this before last night, but we did this event with a friend of Hex, friend of Amplify, Pedram Naveed. Um, and I was talking, you know, he was basically interviewing, it was like a fireside chat and we were talking about this thing that's like, um, it clicked for me, which is like, uh, AI can write for you, like it can write an email draft or or, uh, note. But I really like writing, and part of the reason I like writing is because, um, oh, there's this old saying, like, uh, if you're thinking without writing, you only think you're thinking.

It's brutal for those of us that haven't been writing too much lately, but yeah.

Well, it's, it's a good practice, but it's fair.

Yeah, it forces, it forces your thought.

Hey, in your profession, right, like you, you get excited about a company, at some point you need to sit down and write an investment memo.

Yeah, it is clarifying when the rubber meets the road, right?

Like, do the words make sense as I'm writing them? And I think this is kind of true in other endeavors as well. And, and I think in data it's true as well. Like you can, you can conceive of like an agent that you give a task and have it go do a whole analysis for you. And I think if you haven't done that professionally, you kind of look at that and you're like, yeah, the robots will do it. And, and I, I'm not saying they won't do that at all, but I also think that there's a real thing of like, if you're actually a data professional, and I, and, and I guess I mean like if you're a data scientist or data analyst, but even if you're just someone who sort of engages in analytical work, there's something about doing the tasks and like thinking as you're doing them that is, is very helpful and almost meditative. And, uh, I wonder how this will go in like coding tools, and I wonder how this will go in other things where there, there's like, there's a level of engagement with the task that you want to keep. And that doesn't mean like, like trying to remember the right pandas syntax, like maybe that part goes away, but, um, what do we lose if we don't spend time on the task, like thinking about it? Like that is an interesting thing. I'm not sure I know the answer to.

Yeah. Well, some of the things, you know, when I bring up this example of this demo that I made that looked really cool, but actually is not useful in any way, shape, or form. There are a lot of things that make that not useful. So one is it wasn't sophisticated enough at that time because we just did an API call. To know different columns across different tables and how to join them. I know from, you know, I was at Facebook where we had really, really large datasets and I was at dbt where we had dbt docs. Sometimes columns are not labeled. Sometimes a table hasn't been updated in like 6 years.

Yeah. Or, or, or there's tables with like the same names and column names, but there's from a different schema that wasn't cleaned up and on and on.

Brian Biscoff's amazing, who leads your AI team, and I've heard him talk a lot about evals and, and how critical they are. I'll just ask more broadly, how do you think about creating reliability? I know that one is what we just talked about, which is you have a human on the other end who actually knows this stuff and knows to say like, that looks like BS. But of course you want it to still be as good as, as you can without that human check. What are all of the elements that build that reliability? And maybe we could start with evals. We talked a little bit about the ability to iterate in the notebook itself. So there's a UI component, but let, let's get into it.

Well, the number one thing is context. No amount of prompt hacking gets you around not having the, the right context. And so, you know, in Hex, we've got a bunch of great features to do data curation and management. We've got, um, integrations with dbt Docs, but also dbt Semantic Layer and other semantic layers to pull that information in. We also have like a lot, a lot, a lot of projects people have built. Now we don't, we don't train on that and we're not like cross-leaking or anything, but we can do retrieval on the projects that are already in your workspace. In fact, a really popular thing with our AI features now is like if you ask a question, we'll show you a list of existing projects that might answer that. Like, It's like, we could go dream you up a new thing, but like, here's something from your data team that has probably got the right answer in it. And so, um, it's that context that, that really, really matters. And then like the evals basically test your ability to retrieve on and, and generate the right thing based on the context. And we've recently written a really extensive eval set internally, like hundreds and hundreds of questions that are like real. Things. And this is another example, like we were talking about demo datasets, like the state of public evals for data stuff is really weak. And I don't mean to malign the people doing it, you know, I mean, I really appreciate like, there's like this set, SpiderSQL, really appreciate people who put in that effort to make that out and put that out in public. The problem is it's kind of like this, um, you see some of these that are, um, you know, uh, not deeply informed by someone who's spent time actually like on a data team. I'll put it that way. Where it's like these very simplistic question-answer pairs. And, um, there's kind of this, this thing of like, can you have evals that actually represent what people are asking? And even variants of questions. So, oh, here's the eval for what was our revenue broken up by segment last quarter? And here's the shape of the data you want back. Well, what if someone said segmented revenue last couple quarters or last, you know, it's like, can you get all the variants right? Can you actually understand their intent? Can you map it well? Like there's a lot of sort of very interesting questions in there. Um, and when we're setting up evals and we're iterating on our prompts and we're iterating on our retrieval strategies and we're iterating on our embedding and we're iterating on, um, even like our steps in our pipeline as an example, like using LLMs as judges for results. You have to like have a deep appreciation for all of that. And we've got, you know, hundreds and hundreds and hundreds of customers now where we see that up close and we get to talk to them and work with them on it. And it's very, very helpful.

Tell me if this is the wrong assumption. I assume in the case of when you have these multi-step agents, if you want to call them that, you have both at each step is the right thing, some eval for is the right thing coming out, and then is the ultimate intention at the end evals for the full generation.

Yes. And, and the, the real litmus test is here's a question, here's the shape of the data you want back. Like literally, this is the columns you should be getting back. And, uh, it's less, it's like you'll see these things where like, oh, new version of the model or Claude instead of OpenAI, where it generates slightly different SQL. Well, that's actually, it can be fine actually. Like there's, there's sometimes different ways to skin the cat, but it's like, do you get the same end? Result is the, is the real question.

If context is the most important thing, do you think that there are other ways that AI can aid context today? So we talked about documentation just as an example where this to me seems like an obvious, you can take a bunch of metadata from there. You can create column descriptions every time.

I'm really excited about, I'm really excited about dbt right now, actually. Like, I think they have this opportunity to become this sort of like cross-platform, uh, like I was like metadata context engine. And there's, there's some context DBT doesn't necessarily see, like the actual analysis being built. But, you know, if you look at sort of the context DBT has or could help accumulate, it's really powerful. And I'm, I'm really excited about like the potential for like AI sort of like looking at the queries that are being written in your warehouse or, um, how things are changing or your schema evolves. And like, can it help you just sort of proactively keep your context and your governance in place, I think there's some really cool opportunities there for them and probably a lot of other companies. And I think, I think about, we think a lot about this loop between the data teams and the sort of, you know, they're like canonizing what's true in the world. If you sort of think about their job abstractly, and then people who are who are using that and maybe asking their own questions based on that. And then the data team seeing what's being asked and what's coming back and what's maybe not being answered, um, or doesn't have an answer and being able to sort of like have this compounding, um, loop is like a really interesting one.

How do you think about tool use? Like humans today use a bunch of tools potentially within Hex and they use cells in our notebooks.

And, um, those are just tools. Like it's actually a really elegant thing. Like, we just, we literally prompt the model with tools. Like, here's the cells, here's what they do, here's the function to call. Um, and it's actually very elegant in that, in that regard. It's like the same tools humans use in Hex. Give it to the LLM, here's the caller to get the outputs of it, and look at the outputs. And, you know, you can go construct the next tool you want to use. And it's exactly what I do in Project, you know, write a SQL query, look at the results, be like, oh, that's cool, I wonder I look at this a different way. Okay, interesting. Um, I'm gonna add this cell when I add a visit for that. I'm gonna add some text explaining what these things are along the way.

Yeah. And there's a beauty in constraining what the AI can use as tools as well. I feel like sometimes when you talk about agents, it's like you go do this thing that maybe leads you in a different direction, but here you have them baked into the product.

Yeah, I, I think, um, there's something very interesting there.

You know, we talked a lot about AI in this conversation, but How important do you think AI is to the Hex story? Are there things that you are more excited about or that you're excited about outside of AI right now too?

I'm excited about a lot of things. I, I'm excited about a lot of things. I think, um, we have launched some really great stuff recently to, to sort of have a full-on— you think of as like the self-serve offering in Hex where you have less technical people asking, answering questions of data. That's awesome. Can AI be a meaningful part of that? Yes. Is it great without that? Hell yeah, it's great. Um, and so there's a lot of work the team's been putting into making that awesome. There's a bunch of really cool stuff we've been doing with our compute engine, not just making it more performant, but also more efficient for customers and ultimately help them reduce their warehouse bills. Um, again, like that may seem unrelated to AI. I actually think it's more related than you might think because if you can make it cheaper to query a lot of your data, you can potentially have agents querying that data, uh, like autonomously and not have it run up your warehouse bill to, you know, high heaven. Like, that's good for everyone, I think. Um, even the warehouse vendors actually, because it's people will be doing more and getting more out of the data. Um, and so yeah, there's all, there's all sorts of things, but I, I think I don't think of AI as like this add-on to the product. I also don't necessarily think of it as like the next, uh, like species our product becomes. Like, it's all like, I think there are certain categories where it strikes me that AI might just change how that thing is done. And like the software that we used to do that thing are just never going to look the same. There's other categories where I think it's like, yeah, you can imagine a version of this 3 years from now that you can't imagine that without AI. And it's really great. And it feels really integral to the way the product works. But it's not like an unrecognizable thing. And I, I, I suspect in data, I suspect in data is probably more like that than not. But I also think that, um, the world's changing fast and we spend a lot of time thinking about this and, um, who knows, maybe we'll get back together in a couple years and I'll have a different opinion.

Okay. So we talked a lot about how AI is being integrated into the Hex product. I'm curious what you're seeing with Hex for AI. I spoke to some folks on your AI team, and I know that they use Hex to prototype some of their AI models. And I'd imagine that there's something that AI engineers could learn from data scientists. AI engineers are sort of this new category of title that are coming up. So how are you seeing Hex being used for AI development?

Yeah, we use our, we use it a lot. So like all of our evals and everything is written in Hex. We prototype a lot in Hex, just prompt engineering and prompt hacking and all of that. A lot of demos just run in Hex, like, you know, we're building a new endpoint to handle a certain task and we can basically build a Hex app to mock that. So yeah, it's really useful for that. And, you know, we've thought about a lot of ways to make it more useful for that. It's not obvious to me that the biggest untapped market opportunity for us are people doing LLM feature development, like There's a lot of those. I don't know, like, is it like as many of them as other people we can go and reach?

Yeah. But feature development aside, even the product analytics or data science of how an AI or agent is working, do you have any tips for people? Because a lot of people are deploying AI in production. Not as many people are deep in the data science world like you.

Yeah. You know, we're, we're, we're maybe a weird one to emulate. Like we built our own sort of eval and testing framework called Spellgrounds. And we're able to do that because we build the product we build and it's where we kind of like give it for free a little bit. Some, some of the stuff for free, like, you know, having notebooks and they're scheduled and run, have APIs on them and like we can programmatically edit them. Um, you know, there's a lot of great tools out there that are, that are, are doing stuff here. Um, I won't like name names and pick favorites. Um, so yeah, my tip is everyone should use Hex for as much as they can, but there's also some other great products and I'm not like, I don't think the best AI eval and experimentation solution in the world is necessarily Hex, like out of the box. We've kind of bent it to our will because we build it and we like using our own products.

And then do you have any thoughts on what the data science team looks like once they have these capabilities? It sounds like there's some questions around what the next 3 years look like, but it's the same flavor and maybe some of the tasks, like I don't remember how to do this in Python or remove No, there's a core question here.

There's a core question here no one knows the answer to, but I think we can try to infer some things.

So I thought you had the tarot cards that tell you, uh, those are just on our shirts.

Um, uh, we, so if you think of the job as a data scientist, it just writing code. Yeah, that's going to go away. Um, but I think if you've done this job, which you have and I have, you're kind of like, yeah, I actually think the like code writing part is, is like meaningful, but it's not. It's not, um, the main event. Like a really good data person that goes and talks to stakeholders and understands things and like dives in and thinks about, well, is this actually like actionable? And like, uh, has, has a lot of that sort of organizational context. And I think if you're a maximalist, you can be like, yeah, but the AI will like read your Slack and Notion and, and know all of these things about your org and be everyone's best friend and will be very persuasive and politically savvy to convince the marketing leader that they should do this thing. It's like not obvious to me that that's gonna happen anytime soon. And, um, and I, and I think, I think we're still going to have plenty of data people in a few years. And I actually think they're just going to be way more productive and higher output and higher ROI and probably happier. Um, cause there's a lot of tasks they have to do today that can be abstracted away. Uh, um, this is kind of a funny aside, but there's an interesting, um, thing called Jevons' paradox from, it was like the mid-1800s. And this economist, I think his name was William Jevons, uh, noted a weird thing that felt like a paradox then. And it's, it's not obviously a paradox today, but you kind of have to think back to that era where steam engines were getting more efficient. So these were powered by coal and per, you know, whatever kilo of coal, they were outputting more watts. Um, because we were just getting better at making steam engines, but the net coal consumption in Britain was going up. And he like couldn't figure out why would this be? These engines are getting more efficient. And to us today, it's kind of obvious because it's like people were just using a lot more. But like, uh, the, the, it's what, this is an example of what's called a rebound effect, which is basically the cheaper something, something gets to do, the more of it you get. And it turns out that we just found like a lot of ways to use watts. In fact, like there's kind of this infinite demand for wattage and like power and energy. And like, you know, you and I are talking using, you know, a lot of energy to talk via video, right? And it's like, what's happening now is insane compared to that era. And like, they would have never thought of— and I think about that sometimes of like, what are the tasks that AI will just make like much cheaper, uh, to, to the extent that maybe we don't get less of it, but we get more. And so it's like, if you, if you kind of think of like, um, like the work a data scientist or a data analyst might do gets, gets lesser, do you need less of them? Well, I don't know, that kind of presupposes that there's this like supply-demand balance today that like the number of data people in an org is the number it is because it's like the, the toy example of this is like, okay, there's this many data questions that come in from the stakeholders and each analyst can handle this many. And so we need 4 analysts and they'll, but it's not really like that. It's like there's an infinite number of these questions and things data people can be doing. And it's like. There's a very small subset that winds up getting engaged on. And so I wonder if some days it's like, do you have, do you actually wind up with more data people? Like, wouldn't that be kind of weird? But like, I don't know. I think about that and maybe just the term of like, who is a data person becomes redefined and we can just sort of have a much more expansive sense of who can do these things. And it's not gated on like knowledge of the pandas syntax and knowing how to do Python package management. It's gated on something else. I don't know, but like, I, I don't know that it means that like data teams go away. In fact, I think that's like the least likely thing to happen.

When I think about you, I think one of the Barry mottos I've imprinted in my mind is, uh, are you scared of what I'm going to know? High ceiling, low floor.

Yeah.

I think this is just a capability that like lowers the low floor. So it's just much easier for people to do the base level of what they need to do. And maybe the ceiling as a result goes up because You buy that time.

I think it's very possible. So, um, well, only one way to find out. We should do this again. We'll, we'll check in on my predictions.

We'll check in. We'll do, we'll do this again in, in 3 years, 5 years, 10 years, the next time in person. To wrap, I'm gonna ask you one more question. So I feel very lucky to be at Amplify and that they invested in Hex and that I angel invested in Hex before Amplify, that I've known you for so long, but I actually spoke to some of your friends from Northwestern before. Any of the Hex journey who said you were always a leader and you're bringing in like the best artists and concerts to school. Maybe it's like in your veins, but I'm curious how you've changed as a person and leader since you started Hex.

Yeah, it's a, well, it's a big question.

Um, yeah, no pressure.

Yeah. So I, that's right. In college I, I was like, my big sort of extracurricular thing, which I often spent more time on than my curriculars, was, uh, this group at campus that brought in like all the concerts and speakers of holiday, you know, and it was super fun. And, um, I actually think in a funny way it was like really good practice for this, um, in a way I had no conception of, which is like you have a bunch of people who don't really know what they're doing with kind of a limited budget and, you know, we like, we had great swag and we like, you know, it was a microcosm. Um, have I evolved as a leader? I think the thing you realize as company gets bigger is like communicating is just very important. And there's things that I take for granted that kind of everyone knows and you have to remember like no one knows it or like. Maybe like the small number of people I talk to every day know it, but like you have to really communicate and share in a way that is going to be, um, more legible for people. Um, and, uh, I think just being more mindful of that and trying to do that in a way that's more consistent and present, um, and mindful is, is a thing I'm, I've grown on. I still think I have a lot of growth to do though.

Well, thanks for coming on and, uh, I'll see you in, uh, well, I'll see you before 3 years, but I'll see you for this in 3 years.

I'll see you soon.