Being Pro-Human in the AI Era

Welcome to the Analytics Engineering Podcast, featuring conversations with practitioners inventing the future of analytics engineering. In this episode, I talked to Barry McCardel. Barry is CEO and co-founder of Hex. Hex is an analytics tool that's structured around a notebook experience, but as you'll hear in the episode, goes well beyond the traditional notebook. We're big fans of Hex at dbt Labs and use it for a bunch of our internal data work. In this episode, Barry and I cover some fun ground. We spent the first half talking about notebooks and Hex and data analysis. In the second half, we zoomed out talking about the hype cycle of data science, how AI is different, the experience of building AI products, and how AI will impact data professionals. This conversation was a lot of fun to record, and I think you'll enjoy it. Without further ado, let's get into it. Barry, welcome to the Analytics Engineering Podcast.

Thanks for having me. It's great to finally be on.

Yeah, we— I think we had Caitlin on in season 1. Has she been holding it over your head this whole time?

Yeah, that's right. She's been lording that over me.

Yeah.

Well, she's the real brains of the operation. So you, you started in the right place.

So you run Hex. How long have you been doing this now?

We're right around 4 years, so.

Okay.

Yeah, it was just 4 years ago we started. It was right at the beginning of the pandemic. My co-founders and I got together and it was an interesting way to start, but It's been a really fun 4 years. Yeah.

Do you want to take 30 seconds and just talk a little bit about the Hex story and where you're at today?

Totally. Yeah. So I've been a builder and user of data tools my whole career. I started my career— well, even in undergrad, I was doing research with a bunch of stuff that it was kind of before you even called it data science, but I think it was like proto data science work in this network analysis lab. And I was in consulting out of college doing really gravitating toward these gnarly data projects, mostly using like Excel and Access and like just the best, you know, those are the best tools I had access to for doing the things I was doing was basically building these really complex data apps in Excel. That wasn't a term I really used at the time, but, and then a friend said, hey, you should come over to Palantir. We kind of like do this for real. And I got to spend almost 5 years there using and helping build data solutions for really interesting and hard problems.

It doesn't get any more real than VBA. That's—

yeah, that's right.

Most used programming language in the world.

I will say some of the, some of the code I'm most proud of in my life is probably the VBA, just for the raw, like, difficulty score of working.

Yeah, right.

Yeah. But, but it was interesting then going from actually that sort of very hacky Excel, VBA, Access, secret database on the PC tower under the desk type infrastructure to kind of being thrown into the crucible of the mid-2010s, like first wave of sort of, you know, big data and data science. It was a very formative time. It was very exciting. And at Palantir, I was very fortunate that we were very much on like the bleeding edge of that stuff. And there's a lot of stuff we even take for granted today that we sort of were pushing the envelope on. And it was fun to be part of that. And then I Had a great run there, left, went to a healthcare startup. We were actually— had a pretty kind of like first version of a modern data stack, you could say. We were on Redshift, we were early DBT adoptees in that sort of like initial New York-based DBT fan hotbed.

What company was this?

Was it TrialSpark, which is now called Formation Bio? Yeah, yeah, yeah. And actually, Yolan and me, uh, Breaking Data initially put us in touch. That's how I first met you. Got it. When I was first thinking about Hex. So, um, yeah, they're, they're great. And, um, I got to work with a really great team there. But I think the, the genesis of Hex was actually that experience, which was we had a really great data team and a reasonably modern data stack. You know, we had all these things, we had a BI tool, but our actual data work, like 80-90% of it, wasn't just happening and like cranking out dashboards in our BI tool. It was like deep dive analyses and trying to dig into questions and building forecast models. And we were spending all our time jumping between SQL queries and Jupyter Notebooks and scripts and screenshots of charts put in PDFs of decks sent in emails as our way to like communicate. And so I actually started this journey as a buyer, like I was looking for Hex because I was kind of like, surely someone has built the product to solve this. And it took a few months of kind of bouncing around New York, talking to other people in the data space to realize that no one had built this. And that actually my co-founders and I, who I'd known really well worked closely with at Palantir, we're actually well placed to be the ones to do it. So that's sort of the initial origin story, and that was about 4 years ago now that we got started.

I want to start my list of gotcha questions for you, but I'll soften this a little bit before hitting you with it. DBT is a tool that until you use it for the first time, if somebody just describes it to you, oftentimes your reaction will be like, okay, fine. Like there's a million things that do that. I could do that in this way or that way or that way.

Yeah, yeah, whatever.

And my guess is that Hex in many ways, you have the same experience. So I kind of want to like give you the opportunity to answer this question en masse. Like, what the hell is the difference between Hex and a Jupyter Notebook? Like, why? I mean, we've been using Jupyter for a decade.

Yeah. And I, and I had personally been using Jupyter for a very long time. And I love, I love it still. I have a lot of admiration for that. And I think it's one of, it's a great example of an open source project that built a community and a lot of success and reach. But our insight early on was that popular though they were, there's a bunch of things that were actually holding them back from being like that, that sort of project back from being a really great product and a really great solution for a lot of data teams. And the initial early thing for us was around collaboration and sharing. For a lot of reasons, it's a project that does not lend itself well to that. There's been some attempts to sort of like retcon, um, things like real-time collaboration or other things into it.

But, but for the most part, I remember you folks have thought a lot about like two people can be in the same notebook at the same time, right?

Yeah, it's like you can think of Hex almost like Google Docs for data analytics, data science, right? It's like you can have real-time collaboration, you can have commenting. Now I actually think that, um, and you and I have actually talked about this a while ago, that most of the collaboration patterns and data converge more toward what like sort of software engineering practices should look like, but actually I think for most of those projects, it doesn't mean that you necessarily want to be checking that code and reviewing it in like necessarily Git-based workflows. And actually we just introduced a feature, or we will have by the time this is aired, led by dbt alum Claire Carroll called reviews, which is actually a, um, really awesome feature for doing basically like pull request style workflows in Hex, like to request a review and then publish a new version of a project. So kind of bringing that best practice of like being able to review work like that to collaboration for data projects, especially for people who are for projects that don't make sense to sort of check into Git as code. And coming back to Jupyter for a moment, like IPython notebooks, Jupyter, the way to, to do that was always check all that into to GitHub. In the actual format that is checked in is like impossible to diff, very difficult to work with, and there's no, there's, there's no prospect of ever being able to do things like a really nice project organization, um, a governance, security, and permissioning around it. There's sort of no sense of that built in. A lot of the things you need to actually be working with, uh, and collaborating with data, um, at scale. And that's not to mention a bunch of other issues around the state model, compute. You can kind of go down the list. But I actually think the biggest thing ultimately is not any of that stuff. It, it's that to actually use notebooks, to use Jupyter, Step 1 was like learn computers. It's like you had to install a local Python environment and then install the notebook and then, you know, get all your packages installed and be able to reason about all these dependencies. And then even once you get through all of that, there's only still to this day, there's only 2 cell types in Python notebooks. There's Python and Markdown. And if that's all you're working in, that's great. But in sort of the modern data stack world that we were working in, to the extent that's a useful term, we can come back to that. But, you know, it's all SQL-based and you wanted to be able to write SQL queries to pull data in. You wanted to be able to do things like build viz without having to write a bunch of matplotlib. And I think one of the core insights of Hex, although we don't think of ourselves just as a notebook, was that the notebook format is like a really great format for doing deep dive and exploratory data work. There's a reason it's really popular, but it had kind of been locked up behind this very technical and very intimidating set of things. Like you had to learn all of these things to make it useful for you. And so in some ways, I think one of the biggest insights we had and the thing we made the most progress on early on was just opening this up to a ton more people. So we have built-in SQL cells, we have built-in no-code viz and pivot cells. We've got now no-code calcs to be able to do spreadsheet, like, you know, literally Excel functions in on tables. And so you can actually go end-to-end in a Hex project without writing a single line of code, Python or SQL. And be working on data in a notebook format at sort of infinite data scale without having to think about it at all. And we— one of the really interesting parts for me as I talk to our users and customers is I'll talk to users. I was just on site a few weeks ago at a customer and I asked one of the users, I was like, well, what were you using before? Like, what were you using, a notebook? And they were like, look to me kind of confused. They like pointed to like a paper notebook on their desk. They're like, like this. They had no concept of Jupyter even existing for them. Hex is just this data tool that is this really great format for working with data. And so I think that kind of points to this thing of like, I think there's—

That's interesting. What percentage of your users do you think are coming from the Jupyter world versus are coming from the BI world versus like are coming from the Excel world? How do people find you?

It's hard for us to tell exactly. You know, one interesting— I could almost push back on the framing of that a little bit, Tristan, because one of the interesting things for us is that these aren't necessarily discrete worlds. You saw, we see a lot of people who have to jump around between those things. To get their job done. I think that's kind of one of the pathologies you'll see. It's like, I'm going to run a SQL query over here and download the data to a CSV. I'm going to put it in my Jupyter notebook. I'm going to run that. I'm going to get to some insight. I'm going to download that to a CSV. I'm going to put it up to back this Tableau dashboard so the stakeholders can look at it. And that is like a deeply pathological workflow because the second that they come back and they want to see a different cut or refresh data or whatever, you have to pull that whole magic trick off again. And so in Hex, we kind of collapse that down where you can do that end-to-end workflow in one. Really delightful place that's collaborative and easy to use and very secure, which is ultimately what people are looking for. Now, it is an interesting question you ask because I— it's hard for us to tell exactly, but as far as we can tell, the majority of our editors never used a notebook before, and they may have been aware of them, but these are our most ardent— some of our most ardent users that we meet are not like hardcore Pythonistas. Who are coming to Hex as a better Jupyter. There are people who are mostly writing SQL or doing a lot of point-and-click stuff for whom Hex is just a way better way to work. And that is great because that is sort of our original vision was like, there's a huge opportunity to change the way these workflows really feel for a much bigger population of people than the current tools, including things like IPython Notebooks, were able to reach.

Again, back to the, like, I see everybody sees the world through their own problems. Their DBT is open source. There's a lot of solutions that bake in DBT in some way, shape, or form. You have a version of this challenge, which, I mean, that's a blessing. It's, you know, amazing to write a thing that other people then incorporate into their own tooling. But there are a million products out there, including products sold by the hyperscalers that on the surface, now you've said a lot of reasons for which these are not the same, but on the surface kind of look similar to Hex. Do you run into this education problem at every single customer that you, you start with? Like, here's how Hex is different than SageMaker, etc.?

Some. I mean, it comes up. I think one of the reasons we're reticent to just position ourselves fully just as a notebook is I think that just, just being the best at stacking a bunch of cells on top of each other is not like necessarily going to going to build you a moat. I think what makes Hex special is a lot of other things, and it kind of is at every layer of the stack, right? There's a ton of front end and sort of visual features we have that other products don't. There's amazing things we do with the collaboration and organization and governance layer that if you're just using SageMaker, you're not going to get. There's things at the compute layer and how we think about that and push down compute and run the caching layer we have. The way we've incorporated AI, these are all very unique and we can tell customers about them all day. But ultimately, I think one of the insights is that a lot of the hyperscaler products are actually built for different use cases and needs. The hyperscalers, the, you know, SageMaker, Databricks, Google, those products that they build are really meant to be front-end drivers for compute consumption because that's how they make their money, right? Like they are targeting those products toward people who are doing very large-scale compute workloads. In the case of SageMaker, you know, it's really just a driver for EC2 consumption. And so you can see that when you look at the product and you use it, it's like very uninterested in a lot of the things we're interested in that are important to the people doing those types of workflows we're targeting. Like we don't, we don't necessarily target those things. In fact, we, there's a bunch of features SageMaker has that we don't for those types of like, you know, large-scale compute things that, you know, their target market. So, you know, as an example, like built-in charts, real-time collab, project organization and governance and visibility for admins. These are things that they just are never gonna build because they're not important in that context like they are in the type product we built.

Yeah, that makes sense. Talk to me about market structure in the space that you're in. If you were going to construct a bear case on why not to invest in any tool at the— I don't know if you want to call it the BI, the analytics layer, the front end of the data, the front end of the data stack— one of the bear cases you can construct is that Power BI is coming to eat everyone's lunch. One of the other ones is that there's just a tremendous amount of dispersion And that the areas that value coming together, centralization, are the kind of more infrastructure layers. And that there tends to be at the front end, you know, one of the things that Drew and I always say is there's no wrong way to do analytics. So whatever the workflow and the tooling is that supports your use case and your particular skillset and whatever, but that works against, I mean, you listed a bunch of stuff that Hex is great at that actually feels very infrastructure-y. You're focused on caching. You, one of the magic things about the different cells that you have is that you have a data layer that actually passes data between those cells in a way, you know, whatever. You can talk about it more effectively than I can, but the technology like that is better to invest in when there is a dominant standard where like all the dollars in that layer kind of flow to this one or small set of tools. But not 50 different tools or whatever. Do you think that there is consolidation to be had in this, this layer of the stack?

Oh, totally. Yeah. I mean, I think we see this at every layer of the data stack and not even just the data stack, you know, like any, any area of enterprise technology and software right now is seeing consolidatory pushes. You can look in HR tech, you can look in finance tech, you can look in dev tools. All of it is seeing this, this consolidation push, you know, different types of databases. You know, there's— is there a standalone market for vector databases is like a really interesting question right now. And so you're seeing this all over the place. Some of it is just natural cycles. Some of this is sort of the end of the zero interest rate environment and just tighter budgets. We are personally, I think, in some ways poised to be beneficiaries of that. But this is one of the reasons from very early that we never thought of ourselves or aspired to just be like a really great notebook. In fact, I, I never felt like there was a standalone market for notebooks. Like I remember in my seed pitch, uh, to investors saying there's not a notebook market. And if you don't invest in us, that's fine. But, um, you know, don't— any company that has like note in their name or is saying that there's going to be a standalone notebook thing, it's just not going to be a big enough market for them, even if they build a great product. And I know that sounds ironic. Given that, you know, we hear from users all the time that we've built the best notebook. But for us, that was always an entrée and like an entry point into a much bigger market. Now, I think there's a real question of like, does Power BI or Tableau, just these sort of last-gen dominant BI players, come and roll everything up? I don't know. And it is certainly very hard to shadowbox with like Microsoft and sort of the juggernaut they built. And there's like, you know, like a million analogies or like anecdotes and not just from the last few years. You know, it's funny because we can talk about things that are like living memory for us, like Slack and Teams, but like there's generations of this. I was talking about this with my uncle who was like a very successful tech exec in the 2000s, and he was talking about Microsoft like snuffing out markets that I'd never heard of. And so they've just been very good at this for a very long time. And it is entirely possible that Hex or dbt or any of us do a really, really great job and Microsoft winds up just steamrolling our market. I don't know how to like ward against that other than just trying to do a great job. And it's a big-ass market, so I think that's helpful. Ultimately though, I think there's a lot of consolidation. I think we see it, and we're— we see a ton of customers who are coming to us and saying, hey, actually we've got too many tools and we want to consolidate. And they're actually rolling up workflows within Hex because it's so flexible and because it can serve a lot of different purposes. And I think We're going to continue to focus on how we can broaden the value for our customers and do more for them, because I think that, that, that trend will continue. And if we can keep doing a great job solving more problems for our customers, we'll actually help them reduce spend, consolidate tooling, and bring more people together, which is ultimately the thing we're after anyway.

I know that when you started, it was fairly common for Hex to be deployed inside of organizations that also had a kind of more classic BI tool. And is that still common or is Hex often now the kind of system of analytical record by itself inside of an organization?

Yeah, it's a, it's a really good question. We've seen this start to change quite a bit. We've made a bunch of improvements over the years and especially last year on sort of like the ability to do really nice reporting stuff that's going to look nice, feel nice, little things like just having tabs. Bigger things like, like investing in new caching systems to make sure things felt really great. We didn't do that as like a great, this is going to be a Tableau killer or whatever. You'll pick your dashboarding-centric tool of choice killer because most of our customers do still have something like that floating around. Those are pretty well entrenched. And our general position is like, if those are doing a great job generating your KPI dashboards, let's leave them there. That's fine. And when we actually show people Hex, I think it's pretty obvious that we don't do things like slightly better than Tableau. We actually just do a whole set of things that those types of tools weren't even built for. Like, you can just do things in, in Hex that are just not possible at all. And so I think it's very obvious and visceral to people why we serve— we can serve a lot of different purposes, but we see more and more customers starting to consolidate. Typically they'll keep that legacy dashboarding solution around for a narrower set of things, but they wind up moving a lot of new projects into Hex. I think it's a really interesting question that we think about a lot internally, like, does this de facto make us BI? Are we secretly a BI company? I really hate that term in a lot of ways. I think, you know, it comes with a lot of baggage and I think it connotes this very sort of like dashboard-centric world that is useful for certain things and a complete mess for others. And as much as we want to, you know, help our customers do a better job on a lot of those workflows and help even reduce their overall spend on things by consolidating, I think we just have a much bigger vision and a much bigger ambition for what we can build. And so I don't think of ourselves and we don't pitch of ourselves as like Hey, replace Tableau one for one. We want to come in and help people do sort of net new things or help consolidate a lot of broken workflows that are just not possible today at all.

One more Hex question before we get into industry stuff. You move quickly. There is always a Hex announcement in my email inbox. I generally hate product marketing emails and I unsubscribe from almost all of them, but I have not unsubscribed from Hex emails and I get—

We have good retention.

Yes, your, your, your launch emails come fast and furious. What is coming that we should know about that you're ready to talk about today?

Well, we're working on a bunch of stuff. It's actually— I appreciate the kind words. We've actually been a little quieter than normal the last few months because we've been heads down on some big foundational pieces at kind of every layer of the stack. Some of it we're not ready to talk about yet. A couple of things we've launched recently that we're pretty proud of, pretty excited about are a next iteration of our built-in AI tools. We can talk a bunch about that. I think that, you know, just generally how AI is going to fit in with data is a very, very interesting topic right now. But we spent an enormous amount of time the last year iterating on both the UX and the underlying generation and retrieval engines for that. We're going to keep doing that. Another thing I'm really excited about that I don't think is as sexy to a lot of people, but might be to this audience is, is the stuff around collaboration and governance. Like I mentioned, the reviews feature, there's stuff coming around better governance and endorsement. We've, you know, invested a lot in search, even like bulk updates and changes for admins, all this stuff that as you're actually building, you have a lot of people building with data in your organization gets really, really important. We have customers that have built thousands and thousands and thousands of projects in Hex. And so helping them be able to govern that well, discover the right things, set people up to be able to work with the right data is really, really important. And that's a set of things that I'm, I'm pretty excited about, even though it doesn't always feel the flashiest. But I think it's amongst the most important things you can do as you're trying to build a really valuable product for successively more complex organizations.

Let's talk about data scientists. You said something like I was a proto data scientist early. Is that what you said?

Yeah, something more like the— it was like 2010, I think, was when I was writing a bunch of these like scripts in— I think it was R, but we also were using like Stata. and it was work that today I think anyone would call data science, but at the time I was just like an undergrad grunt helping with a bunch of like data cleanup work in this research lab. Uh, and then got to kind of trace the whole arc of data science becoming a thing as I was at Palantir, where, you know, that's what a lot of people were looking to us to deliver. And now, you know, several years later, I think seeing how that's sort of evolving in this new era has been really interesting.

I'm learning about this as I watched the modern data stack hype cycle. There's the data science hype cycle. And I don't think anybody is saying that the kind of techniques that we were using in 2015 are any less relevant today than they were in 2015 at solving these set of business problems that they were good at solving in 2015. And probably we've gotten better at solving those problems and more mature tooling and data to all of this stuff. But data science is not sucking the oxygen out of the room in the way that it was back then. I mean, the Analytics Engineering Roundup was originally called the Data Science Roundup because, I mean, let me tell you what the SEO was like on data science back in 2015. It was like quite—

yeah, that was the place to be, man. That and big data. If you said either of those, you were in.

Totally.

Yeah, I said a lot of that in 2015.

I'm curious about the identification of the professionals. Like, do you think that quote-unquote data scientist is still a job description in moving into the future, or are we just all doing data analysis and we're using different tools, methodologies?

Yeah, I've thought about this a lot. I think data science— the very weird thing happened that I still don't fully understand in the mid-2010s where I think you had this, this sense of possibility around these more advanced techniques. And it kind of came out of the tech world. I think a lot of it, you know, came out of like the Google as, you know, sort of this institution that was doing a bunch of these sort of more advanced data techniques. And that started breaking out. I think it was DJ Patil who might have been like the first person to coin the term data science. And, and that really caught on as a, as a hype cycle, as you said, in, you know, a lot of corporations where they're like, oh, we want to do that type of like advanced data work here. And, um, that term became very in vogue. And the joke for a long time was like, a data scientist, just someone doing analytics in Python. It's just like, you know, it's— and there's truth in that.

To me, the height of it was when Lyft literally wrote on their corporate blog, we've retitled all of our data analysts to data scientists, and we're paying them the same, and they're doing the same shit, but we've heard that they're happier now.

Yeah, well, exactly. And it's— you invent this title that, you know, sort of elevates people. But I think ultimately what it did was really model some things. I think there's actually just kind of like two parts to the data world generally. There's analytics and ML engineering. And if you kind of take a representative sample of data scientists, 7 out of 10, 8 out of 10, I don't know, something like that, are actually doing analytics. And what I define analytics as broadly is you are trying to influence a decision. And that can be, you know, through a one-off analysis. That can be you're building a dashboard to help people look at data to decide things in their given day-to-day. You could be doing something with a simple technique like literally just rendering a chart or something really advanced, but ultimately you're just trying to inform a decision. And I think that's basically analytics. And you can do that. That is a really broad and wide and really important sort of discipline, as you have championed as well as anyone in the world. On the other hand, you have machine learning engineering, which is really about building a system for making predictions and inference. And that can be really simple, like a really, you know, simple little linear regression all the way up through these amazing new techniques we're seeing. But I think data science as a title wound up kind of getting both of those things crammed into them. And when they're actually very discrete and different and the workflows look different, the deliverables look different, the skill sets look different. And what I see happen now in our customers is, is this sort of progressive delamination where people who are doing machine learning engineering are just calling themselves machine learning engineers. And for no small part because it puts you in a different pay band if you have engineer in your job title, as all the analytics engineers out there may or may not be realizing. And then the people who are doing the more sort of advanced analytics work, maybe they're kind of retaining the data science title, but it's really kind of returning to that focus. And, you know, I think ultimately the title has maybe outlived its usefulness. And that might sound funny as someone who at some level is like, peddling tools for data science, but even within our own user base, like, I don't see a lot of distinction between the workflows we see data analysts and data scientists doing. In fact, part of the thesis of Hacks is that these people are fundamentally doing similar things and they shouldn't be balkanized into different tools. And, um, friend of, friend of dbt, friend of the pod, Emily Shariot, I think actually had a great talk on this. I think it was at Coalesce or maybe Data Council. Yeah, this was a while ago, the last couple years. Yeah, maybe a couple of years ago now. It was like, I think she was called like Down with Data Science or something like that.

But that felt spicy then. And I feel like that has essentially played out.

I think she was very prescient on that. And I remember going up to her after the talk being like, that is exactly what we are seeing as well. Well, I think her point was, was as I've, as I've been thrusting at, which is like most people who are data scientists are just doing analytics. And actually by, by making this other title that feels, you know, in the Lyft case, as you pointed out, sort of like this just kind of titulature to make someone feel special and exotic. You're actually almost like downgrading the practice of analytics. And I think she kind of had this call to like, let's just all be, let's be analysts and let's like really up the importance of that and the salience of it and acknowledge that there's growth paths even within that discipline in that field. And I would love to see that personally. I think that would be really exciting.

I don't read as much of this kind of stuff these days, but one of the things that used to drive me crazy was posts by new data science practitioners. And frequently they would've gone through some type of coursework, bootcamp, whatever, that would've skilled them up for this job. And the bootcamp would've been very focused on a certain type of techniques, statistical techniques. And then the job search process would have them focus on these techniques, and then they would get into a job and they'd realize, I'm like solving data analysis problems. And then they would be angry about that.

Where's the random forest modeling?

Right, right. And I can understand that because they were trained to believe Thing A, and then they got actually into the world and they, they got Thing B. And that's, that's frustrating when you make real life decisions around it. But for, for people who had been in the industry doing the work for a long time already, we all knew that in fact you were not hired based on the particular set of statistical techniques that you deployed the most frequently. Like, you were hired based on your ability to solve a certain set of business problems and do so kind of doggedly despite all of the million obstacles that would be in your way.

Well, absolutely. And I think what's interesting is these people will come out and be like, they'll be very focused on the technique or like, you know, they almost be just trying to force fit some really exotic technique into a very, you know, run-of-the-mill business decision that needs to be made. And I think there was a lot of exoticism. There was a lot of sort of fetishization of like, how complex can you make something when in reality you just want to answer a question and make a decision? And that, that may or may not necessitate something like that. We saw a lot of this at Palantir in the mid-2010s. We had clients come to us all the time being like, well, I want to do like, we want to do some machine learning or we want to do like a really advanced predictive model to say what our inventory should be next year. And you'd ask them, you'd be like, do you know what your inventory is right now? And they're like, well, no, of course we don't because, you know, that's, that's like split between 8 different systems and we, we won't know that until 2 months from now because of this and this and this. And you're kind of like, Okay, well, I've got an idea. What if we start with helping you understand what your inventory is right now, and then we can worry about the predictive stuff. And I think you kind of— people wanted to like skip over the hard work of just like counting things and get into the like magic Oracle, tell me the future stuff that, that sometimes is possible and useful, but, but more often than not is actually like over-engineered at best. So, you know, I agree with that and I think we maybe, I think some of that has faded out. I think people have a deeper appreciation now for just good old-fashioned counting stuff and making decisions based on it. It sounds boring, but it's super important.

We have just been talking about how the industry gets obsessed with techniques because techniques are fun to talk about and they're very sexy. But we, you know, we're sticking out the position that business problems and good old-fashioned counting stuff is important. But let's entertain AI for a second. The most recent technique that we are all fascinated with is generative AI and large language models. And I think that maybe the interesting question here is, how similar is AI today and machine learning in 2015? Are we going to look back on this in 10 years and say, "Hahaha, remember when all we wanted to do was hit this nail with this hammer?" Or is there something different about this particular—

I think it's very fundamentally different. You can look at them and say that the hype cycle feels similar, and we can talk about ways in which that's true. But having lived through and been pretty deep in the trenches on both of these hype cycles or technology waves, they, they feel very, very different. And the reason why is the ML wave in the mid-2010s, it required an enormous amount of work to get things to work. Like most ML projects in that era wound up as shelved R&D. And the question I would always ask, even when I would interview candidates who sort of had experience with those types of projects, I'd be like, is it in prod? Like, is it serving predictions right now? And I don't know, 80, 90, I don't know, but the vast majority of projects that people would do, especially in sort of these big enterprises, would never make it there. And you had a clutch of, you know, typically San Francisco-based startups You know, the Ubers, the Stitch Fixes, the Pinterests of the world that were, you know, Facebook and Google obviously too, that, that were deploying this stuff at scale for very specific problems. But, you know, the vast majority of things just didn't work and it was very difficult. Contrast that with where we are today with large language models. Anyone can go to ChatGPT right now and experience it working live. And there are a lot of things that aren't working. I do think in many ways we'll look back at the current vintage of models as remarkably primitive. I think they are remarkably primitive. I think The way we interact with them is remarkably primitive. You have to like yell at them in very specific ways. It's like you're, you're lecturing like a rambunctious 7-year-old sometimes when you're doing like prompt engineering. Like you will not reply.

You're a rambunctious 7-year-old.

You're a rambunctious 7-year-old and you will not reply with additional markdown when I ask you this question. It's, it's ridiculous. And I think we'll look back at this and be like, you know, well, that was weird. But I think the fundamental technology works. I think it works well, and I think there's a bunch of applications that it clearly does a great job of. And the, when one of the first that I think people got broad exposure to even before ChatGPT was GitHub Copilot. I think, you know, if you're doing a task that is fundamentally about writing code, these things can write code pretty good and it's not perfect code. And I think there's a very specific set of questions within analytics of like, can and how and will it be able to write sort of like perfect SQL? To query the data in the exact way you want. But as someone building a product where people spend a lot of time writing code, SQL, Python, other languages, it was very obvious to us even before ChatGPT that this is something we wanted to incorporate. And we see it get used and loved by thousands of our users now, the, the sort of magic features we've built in. And so in that way, it feels very real. Like we have some PhDs here, but we don't have like a massive fleet of PhDs and a lot of infrastructure to deploy this stuff. We're hitting these endpoints through APIs that exist and we can just pay for it with a credit card. And in that way, it is incredibly materially different than, than 10 years ago. And I think that's what makes me so bullish, which is ironic because we actually, when we started the company, I think you could characterize my co-founders and I almost as ML skeptics. Like, I think our Palantir experience in some ways and going through that hype cycle in the mid-2010s made us very skeptical. Like, we saw so many times where you were trying to You had a hammer and you wanted everything to be a nail with these really exotic techniques.

We just saw the Dr. Jekyll and Mr. Hyde of Barry on this topic right in the last 20 minutes. This has come out through the conversation. I was ready for you to be more cynical than you are on AI, but you're making a strong case that this is different than the wave before.

It is. And it's easy in this moment to point out specific quirks with GPT-4 or other models or the way we're deploying it. But I still think we're just so early in understanding this stuff. Even, even the UX, like how should it be incorporated into our workflows? So many people are just sort of saying, look, well, we'll just build a chatbot for this and a chatbot for that. And like, maybe there's places where a chat is a really good UX, but I don't— I mean, would it be depressing if all of our software just kind of progressed into being chat? I, I don't think that's right. And even within Hex, we were thinking about this. Every day of like, what, how does this actually fold into your workflow in a way that's helpful and useful? And in some ways, you know, what we talk about internally is sort of it's okay to live in the future a little bit. There are things today that the models don't get right or that they feel really slow. And we're making a choice with some of what we're doing to embrace that and say that's okay. We're going to build for the future because even since we started building with this, inference costs have gone down orders of magnitude. Speed has gone up an order of magnitude. Accuracy has gone up. Many times over. And you know, that's like in a year and a half. So if you kind of extrapolate forward, you're just going to keep seeing these gains. And I think it's okay to be skeptical of certain things today, but I think you have to be very specific on what exactly you're skeptical of and whether that's a short-term, like, hey, we're still figuring this out, or something more structural.

We've had some similar experiences shipping, designing, building products that leverage AI. It turns out that it is just much easier to build an AI driven feature inside of a product than it is to build a machine learning feature inside of a product. So here's the thing that makes me really bullish about specifically Hex and dbt being able to do really effective AI code gen. The place where AI does not do a great job today is where there is really large unstructured solutions. Like, so my sense is that AI can pretty reliably write a program that has well-defined input, well-defined output, and is like 100 lines or less. But the more code it has to write and the harder it is to define what good looks like on the beginning and the end, it drifts. But it turns out that a notebook cell or a DBT model or a DBT test spec, these are very well-understood things. You can clearly define success, and they're like modular enough so that you're not asking AI to generate so much code. I think that oftentimes the hardest problem of getting a good response is structuring the problem in a way where you can expect a good response. And I think both of our products do do that.

Yeah, totally. I think that one of the things we really discovered and appreciated early on was that the sort of naturally modular cellular UI in Hex, the sort of notebook-inspired original UI we wound up with, it lends itself very, very well to these types of things. And we, we actually under the hood, you know, among the many things that we did from a product perspective that goes beyond what other notebook products have done in the past was underneath Hex is actually every Hex project is an execution DAG. We do static code analysis on all the code in your project. We actually execute the code based on the dependency graph, which fixes, you know, we did it because it fixes a lot of problems with state management and a lot of like reproducibility issues you typically have.

You could get some weird shit happening in Jupyter. Back. Yeah.

Yeah. Well, you can, you can, and X solves a lot of that and it's more performant and all that. We built that well before GenAI, but it pays this massive dividend because when you're asking something in our AI features, you're, you're asking yourself, we actually can pass in, you know, the upstream context and understand the upstream dependencies of the code. And, and we can have ways to give more sophisticated context and structure to what you're trying to do. So yeah, we have found that to be really nice. I think there's other dividends on the UX side for us. Just even, even if you look at something like the way we're able to sort of incorporate it into the UI in many ways kind of brings the best of chat where you can sort of sit in a prompt bar at the bottom of the project, just type something you want and watch it generate new cells in a much more functional UI. You know, I don't think one among the many reasons I don't think data work all, you know, just evolves into just being chatbots is because you don't want to like that. That's actually not a highly functional way to be able to interact with the work you're doing, you want to be able to point and click, you want to be able to type, you want to be able to change things. You don't want to have to do every, you know, change the chart colors via prompt.

Multimodality.

That's— and that's how we describe it, sort of the multimodal front end for this. So you can jump in and out of editing or generating with prompts at any point. And I think that the notebook format and UI makes a lot of sense for that. And also, as you said, I think allows us to structure the context for the models in a really clean way. So You know, we're still early. We've done a ton of work on this. We're so early, I think, in probing that frontier. But we have a very clear vision for how we want this to fit into the workflows of folks. And what we talk about all the time is, you know, this is built to augment, not replace human insight. I am fundamentally bullish on data practitioners in this new era. I'm fundamentally pro-human. I think that is very different than a lot of the rhetoric and hype you hear. But we, we think that's the right approach from a product perspective. And we're doubling down on that.

There's a lot of focus around AI in the consumption layer. There's data catalogs that have AI interfaces to help you understand your metadata better. There are multiple different kind of AI-powered analytical experiences. One of the things that I'm really interested in is, and, you know, of course, this is where I would be interested in, but certainly in the data pipeline world, there is the help make me a data pipeline. Cool. But the other thing that happens in data pipeline world, and it probably happens a lot more often than you build new pipelines, is that you're just constantly debugging problems with existing data pipelines. So one of the things that I am really interested in is the ship me a ton of metadata and I will figure out for you the underlying causality that caused this failure last night.

Yes. I'm so excited for this.

I think this is 100% tractable. And I just, right now, I don't think that assessing causality is the place where LLMs today are the strongest, but I think that you can get them to do it in certain cases. Do you agree with this? Yeah.

Well, we can talk about truly assessing causality. I think you, how these things work is even very interesting. You and I are probably not the right people to do a deep examination of LLM structure, but What we've just found ourselves is even things like we have a feature that's called Auto Fix, which is if you have an error in a cell you're running, we just will suggest that we can fix it automatically for you with AI. And actually on the backend, we're—

Can you say anything about how you're getting the model to do that? Is it just a really clever prompt? Are you like, what are you doing?

Yeah, it's prompt engineering. So we, so what we do is we take, I mean, at the high level, we take, you know, we've got a lot of prompt templates we use. We, in that case, we have a prompt template for Auto Fix. We'll take the code from that cell that's executed and errored. We'll take the error message itself, and then we'll also take upstream context. As I mentioned, you know, parsing upstream in the DAG to pass the context of what's actually happening upstream in the project. And you pass that through to the model and with some high percentage, not 100%, it's important to note, but with some high enough percentage of time, it's going to take that information and go and write you a new version of that that will run. Why does that work? How does it reason about that? Can it infer the causality between a bug in your code and the actual error message? Like how these things work under the hood is a little bit spooky.

Almost. It doesn't really matter.

Yeah, it almost doesn't matter. It's just weird. It does. It's an emergent property. And, and we know through our experimentation that that prompt template generates a response that more often than not is fixing it for people. And that's how that feature works. And I think in some ways working with AI is so strange because it's so stochastic. It's like, very, um, you're used to software engineering, very formalistic way of working. And so it is a little weird that sometimes it's like, we don't know why it works, but it works. And there you go. Like in most software engineering, you would not tolerate such an answer for why a feature works, but we do that. And I think like you could, you can imagine in that case you laid out, Tristan, of like building data pipelines or maybe like preparing data pipelines. If it's happening overnight or if at the very least if it's happening where I'm not having to stare at it as a user, you can also just try multiple times. You could kick off 10 different prompts and try to fix it and see which ones get the tests to pass. And yeah, you may run up inference costs or database query costs, but like at some level you can— I— we've even seen this in some things we've experimented with. You almost like brute force things sometimes and sort of embrace this stochasticity of it. And again, it feels a little weird. It feels like you're throwing a bunch of stuff against the wall and seeing what sticks, which is usually not how you approach an engineering task, but It is, it is almost like very fun to have this different way of working that kind of challenges assumptions.

I think it's going to be completely tractable to say, figure out what the problem is and submit a PR to fix the problem.

100%.

I don't want to actually look into data pipeline problems. I want to review PRs that solve them.

There's a massive percent of these pipeline problems could be fixed effectively on autopilot and have someone review it, have someone merge it in. Now, the really interesting question to me, Tristan, is, you know, a very— you can draw a very clean line between you and I on this, this conversation right now, getting really excited about that. People hearing that and saying, well, my job is fixing data pipelines. What do you mean the robot's going to do it for me? What's going to happen to my job? And that's where I think you get that. You get a lot of people out marketing, you know, AI data scientist or AI data analyst as products. And I think it's kind of this implicit assumption of of, hey, these LLMs can write code. Okay, there's a bunch of humans over here who are writing code. We'll replace the humans with the LLMs that can write code. And I fundamentally don't believe that. I think for a bunch of reasons. I don't think anyone wants to be manually going and fixing data pipelines because of upstream format changes. That's the worst part of anyone's workflow.

One of our company values is we believe in moving up the stack. And, you know, one part of the text is there's always more and more valuable work to be done. And 100%, no one pays anyone's salary to fix data pipelines. They pay people salaries to solve business problems.

Well, that's what they want them to be doing. And when they find out that they're actually solving these data pipelines, everyone gets frustrated because it's like this, this is not what we want these people to be doing. I think anyone who's thinking that these things will reduce the sort of net count of people in data analyst or data scientist or analytics engineer roles, I think is missing the point. I think that presupposes that there's some finite demand for insight, right? That like the number of data scientists or data people you have in your org is like—

We know sufficient things. That is enough. Thank you.

It's like, yes, we have, we have 100 questions a month that come in. Each of these people can answer 20 questions a month. Well, then therefore hire 5 of them. That's what we need. No. What company are you aware of that is like, yes, all of our questions are being answered with this many people? No, no one. I think there's an infinite demand for insight and an infinite demand for data work. And if you make it cheaper and faster to be able to do that, you're going to get more of it, not less. And I, I'm super, super bullish on that.

You know, not to go all like pot smoking mode, but like it's a part of the capitalist mechanism that like we're all out here.

Bring it back to college, man.

Yeah, every company is out there competing in an open market to be the best at whatever it is that they do. And part of the way that you are the best is that you are the smartest, you make the decisions the best or the fastest. And if somebody else is doing that better than you are, then you either are going to increase your capability to do that or you're going to fade away. So no, there's not the 100 questions that once you know them, you're done. You need to know more than the person next to you.

100%. And I think it's a very scarce mindset, very sort of short-term thinking point of view that that's— that an LLM being able to take on some of that work will will wither away the need for this. And I, and I don't think that even people want this. I don't think stakeholders, you know, the people actually going and asking the questions of the data team, want to just be like typing a question into a chatbot and getting an answer back. Like, if all you think that data analysts or data scientists do is write SQL or write Python, then yeah, I can see how you would think that an LLM could replace them because it writes pretty good code. But if you've actually done these jobs and you appreciate what makes great data practitioner, you know, it's not like the writing the code is actually kind of in some ways it's the least of it. It's like, it's like there's a ton of other things that go into that. And maybe like GPT-7 will be able to build deep trusting relationships with stakeholders such that it can fully displace the data team.

But I just— work the social graph of your organization like, ah, I need to influence this person to do this thing.

Maybe we'll look back at this conversation and laugh because we'll be like, we just missed the point. But I think a lot of the folks out there marketing like AI data scientists or AI data analysts, they haven't done these jobs. They don't understand it. And maybe, maybe I'm the idiot, but I myself personally and Hex as a company, because I run it, are taking a bet on data teams and data practitioners. And we actually think there's going to be more of these people, not less. Fundamentally, I'm like bullish on this golden era of data teams.

I'm with you.

And that's what we're going to keep pushing on.

So yeah, let's— I think that's a good place to leave it. Last question. What do you hope is true of the data and analytics industry over the coming 5 years?

I hope— I think the promise that data is going to drive better decisions and make organizations better and all of that, I think, has been a long time coming. And I think you could look at it cynically and be like, well, we've been promised this for a long time. I actually think for many reasons, the maturation of the modern data stack, the genesis of AI, and a deeper appreciation of a bunch of how this work gets done. I think we actually haven't— the next 5 years is the opportunity to actually have that feel real for more organizations. I think even just the thing we were talking about, like getting people out of the manual, tedious work, spending most of their day just fixing a data pipeline instead of going and actually helping drive a decision or answer a question. Like there's an enormous potential to unlock there of actually helping these people do real and valuable work versus just plumbing. And that is really exciting to me. And I, I am bullish on that kind of fully potentiating. That's what I'm looking forward to, and that's what we're in our own way trying to contribute to. If we're successful, we'll have built maybe a product that is helping accelerate and deepen people's ability to do that. That's what I'm optimistic about right now.

Barry, thank you so much for being on. This is a lot of fun.

Thank you.

The Analytics Engineering Podcast is sponsored by dbt Labs. I'm your host, Tristan Handy. Email us at podcast@dbtlabs.com with comments and guest suggestions. Our producers are Jeff Fox and Dan Poppe. If you enjoyed the show, drop us a review or share with a friend. Thanks for listening.