[COALESCE] The Modern Data Experience w/ Benn Stancil of Mode

Talk by Benn Stancil (Mode co-founder & Chief Analytics Officer) at Coalesce 2024 arguing that a centrally governed metrics layer is a discontinuity that ushers in a new era — the modern data experience — built as a platform where governed metrics power diverse last-mile experiences. He traces the evolution from monolithic BI (MicroStrategy/BusinessObjects in 1989–1990) through the modern data stack (Redshift, dbt) to today’s fragmentation of metric definitions, and explains how a centralized metrics/semantic layer lets consumption tools focus on experience. Mode is building metric-first reports where users pick governed metrics, view rendered SQL, and extend analyses into notebooks and visual explorers. The transcript finishes with a live Q&A covering integration, missing pieces (embedded/operational data apps), and performance/latency observations.
Questions this video answers
1989.
1990.
May 28, 2024.
More than 10,000 attendees.
Chapters
Welcome to the Analytics Engineering Podcast, featuring conversations with practitioners inventing the future of analytics engineering.
This week is one of our favorite weeks of the year because it's coalesce. Coalesce is DBT Labs online data conference dedicated to the advancement and practice of analytics engineering. We have nearly 100 speakers giving talks across five days, and this year we have more than 10,000 attendees. We're broadcasting a few talks on the podcast so that our listeners don't have to miss out on great content. You're about to listen to co founder and Chief Analytics Officer of Mode, Ben Stancil. We've had him on the podcast before, so you know he's full of hot takes and unique insights. In his talk, Ben walks through what he believes the next evolution of the modern data stack should look like. And my guess is he's spot on. Let's hit play.
All right, all right, all right. Who else is having the best day ever? I know I am. My name's Anna. I'm heading up community here at DBT Labs, and I'm just blown away by the sessions today. I'm hosting this session and Mila Page, who's our esteemed developer advocate, is your chat champion today in coalesce Mode, of course. The title of the session is Metrics and the Modern Data Experience. And we're joined today by the one and only Ben Stancil. You might recognize Ben from the classic Friday data fights on Twitter or references to substack. Yes, the very same ones that are on the bingo board in his spare time, of which he has a lot. He's also the co founder and Chief analytics officer at Mode. Things you need to know about Ben, he will fight you about coffee cake. He's from one of the Carolinas. Can you guess which one? And he will always have a better metaphor than you. That's just facts. So today's talk is part of a conversation that Ben and a lot of other folks in the community have been having for several months now. You heard from Drew a little while ago on metrics. Actually, a lot on metrics and 5,000 years of metrics. And Jeremy, just before this session, shared how all of this is going to work in 1.0. And I'm very excited today to see Ben bringing all of this together. A gentle reminder that all chat conversations are happening in Coalesce Mode. The memes are already starting. Don't miss out. It's on the DBT community. Slack. We're going to share relevant links in Slack as Ben is talking. So you won't miss any of the references. And of course, slides are going to be available in the email that Lauren's going to put together at the end of the day. Please do say hi, introduce yourself in the slack, do ask other attendees questions, make comments, react at any point in the channel, and do use Slack threads to keep the conversation tidy. After the session, Ben is going to hang around, answer some questions, so throw them in the chat anytime and we'll make sure to come back to them. Oh, and one more thing. It's not Friday yet, but the best hot take during today's session is going to unlock a very special hot take back from Ben. Let's get started. Over to you, Ben.
All right. Howdy. So I just saw Drew's talk for the first time a couple hours ago. So aside from these screenshots that I took that you may see, any similarities between this side, including our apparent shared paralyzing fear of snakes, are entirely coincidental. So just a disclaimer on that. Okay, so a couple of days ago at the start of this conference, Tristan posed a question along the lines of, what are we going to be talking about today that we're going to remember in 100 years? And so like on December 8, 202121 or 2121, hard thing to say. What are we still going to be talking about? Like, will any of this stuff still matter? And so I don't know if we're going to remember today 100 years from now. We've got a lot of infamous history to compete with on December 7th and 8th. But I do think that today will matter, and I think specifically it will matter because of what Drew just announced, that that announcement marks the end of an era and the beginning of another one. So on this like decades long or millennia long journey, as Drew put it, the data journey that we're on, I think today actually marks a rare discontinuity. So in other words, basically I want to say that the metrics layer is the future of analytics. In order to do that, I'm going to follow a classic John Oliver narrative arc. I want to talk about how we got here. I want to talk about what's changed with what you announced, and I want to talk about where we're going to go next. And so to do that, I want to talk about initially to start with where it all began, which is the ancient Egyptians. But Abdogasar by his sense of history is not nearly as good as Drew's. So my history actually starts in 1989. So 1989, MicroStrategy was founded. And a year later, in 1990, BusinessObjects was founded. And back before this was back before business objects was SAP. Business objects, it was acquired by SAP. And back before MicroStrategy was a Bitcoin company, these two companies were pioneering BI companies. And they ushered in what I would call the age of big bi, where there was big BI tools, these big monolithic BI platforms that really changed the way that we think about data. And roughly, here's what they did. Companies would have data, they would store it in a database. They would store it raw. It was messy, represented by these messy purple tables here. But it wasn't just a database. It was stored in a slow and expensive database, so people couldn't write a lot of queries against it. There wasn't that much you could do that was valuable. And BI Tools came along and figured out a way to fix this. So they did it in a couple ways. First, because that data was messy, BI Tools developed a way to model that message. And these were the models that would define relationships between tables. This is what that actually looks like in MicroStrategy. And this step would turn messy data like stripe logs or things like that into something like dimension purchases, which is a clean data set where you have one row per order. You have, like, which item was bought, how much it cost, was it bought on a gift card, these sorts of things. And then you would have tools that would define how to actually calculate metrics on top of that clean data. So you could define, say, revenue from dimension purchases, where you would add up the cost of every item, maybe remove things like store credits, remove taxes, all that kind of stuff. And this is again how you would define metrics in MicroStrategy. And then on top of that, people would build reports and companies could suddenly do a lot more with their data. And so more stylistically, this is what it looked like. BI Tools would end up taking having modeling raw data and putting it into a data model. Out would come these nice clean tables. Metrics would then get computed on top of that. So clean tables like dimension purchases could be turned into revenue. And then people would start to put charts and reports on top of that. Now, as Drew alluded to, there are obviously a lot of limitations with this. Everything had to be pre computed. It all had to be loaded into this data model. The things you could see were pretty narrow and the data had to be pretty small. You also couldn't really do a whole lot with this data. The data in the warehouse, because the warehouse was expensive and slow so there wasn't actually much value in this. But the big thing that BI did do is had one huge benefit which which companies had a really consistent and reliable view of their businesses that anybody could look at reports and get the information they needed to make decisions using well governed data and well defined metrics. And so this was a tremendously valuable thing. And to explain why, I want to talk about cubes, but not this cube, Rubik's Cubes, but not this Rubik's Cube or this Rubik's Cube actually, but this Rubik's Cube. So this is a seven dimensional Rubik's Cube. This is something that people actually try to solve. Shout out to my former coworker Nan Ma, and also data scientist for being the third person in the world to solve this thing. Only 11 people ever have. But when you're trying to solve a Rubik's Cube like this, you can't actually do it obviously with a physical toy. There's no seven dimensional toy that you can play with to try to solve this. The way you have to solve it is code. And this is the code that defines a actually four dimensional Rubik's Cube. So you can imagine what a seven dimensional might look like. And so solving this problem is plenty hard enough as it is. But if there is a bug in your code or one of the faces gets messed up, or your computer program that is modeling it crashes, you've got a real problem because you can't go back on the physical cube to say, hey, what does this thing actually look like? And if that code breaks, then the cube is broken. That a seven dimensional Rubik's Cube only exists as data and code. And if that code doesn't work or it changes unexpectedly, you can't actually solve this problem. And so companies are like these Rubik's Cubes. All decisions we want to make depend on the metrics that are defined in this code. And so if we want to make decisions about revenue, then we have to have good definitions of what revenue actually means. And so there's a lot of decisions that are all based on our understanding of revenue for making decisions about which product variants to build, making decisions about how much to charge for that product, about what to do when an unexpected crisis hits, or need to evaluate our ML models to making sure we're not going to, we're not going to avoid a catastrophe. We have to be able to understand what revenue is. And so that is really what BI solved. And so we solved this problem decades ago and we all, at least now have a shared understanding of revenue or the other metrics that define our business. And then around 2012 things started to change. So one of the big reasons, as Tristan has talked about in the prior blog post, is that Redshift came along and what this did is it turned our slow and expensive databases into ones that were fast and cheap. And that enabled us to do a lot more with this data. And so in addition to be able to continue to do reporting and things like that, we could also do analytics. We could do operational, have built operational tools. We could apply AI to this data. And of course DBT came along and pulled out a lot of this data modeling into its own layer. And so DBT ended up cleaning up a lot of this data representing. And so this really ushered in this kind of second age of data, which I would call the age of the modern data stack. And this has been a tremendously beneficial thing. It's mostly been very, very good. We get all these very powerful things that we can do with data that we couldn't do otherwise. And Jason Ganz highlighted a lot of this yesterday. The impact we could have with data in a post analytics engineering world, which is kind of the same as a modern data stack world, is way higher than it was before. We can make way smarter decisions. We can do a lot better about how we think about marketing our products. We can be smarter about operations such as using AI to manage fleets of airplanes. All of this stuff is great, it's powerful, we love it. But on one of the more fundamental problems, on the problem of reliably defining this Rubik's Cube and making sure we're all sharing the same reality when we're trying to solve these problems, we've taken one step forward and three steps back. And the reason for that is because these metrics are now scattered all over the place that every tool in this stack has to represent how have to figure out how they actually want to represent that Rubik's Cube. There is no central definition for that Rubik's Cube. It has to live in every individual tool. And so you see this across different tools in the stack. So Looker has LookML in mode. We do this through a thing called definitions where it's a way to represent what these metrics look like. Observability tools like BigEye have metrics. AI tools like Sisu have ways to define metrics. And of course there is a wholly number of Excel spreadsheets and SQL queries that live out in the ether that define these metrics as well. And so As a result, what happens is all these tools, to borrow Drew's language, we end up arguing over which latitude we should be using to measure things. And so the companies that are using them end up playing this endless game of whack a mole, trying to get everything to line up. And so this was something that was put really well by Ali Godsey, the CEO of Databricks. And Databricks is building like one of the most technologically advanced data tools out there. And at Future Data, he said that despite all that we've built, despite how far we have come in all of this technology, most of their customers are running around trying to figure out why somebody's numbers don't match somebody else's. All their customers spend time bickering about whose numbers are right. And so really despite all of this, we're really just reliving 1989 until metrics layer. And so over the last year we've seen a lot of activity in this space which has obviously accelerated further in the last month. Lookers last month announced the opening up of a universal semantic layer. And then obviously today DBT joined the party with Drew's Talk. And so to me, what this talk, what Drew's Talk and really DBT's commitment to this represents, is that we're now solidifying a new architecture that looks like this. Like this is the architecture of the future where data models and metrics are centrally governed. That dimension purchases is not just universally accessible, but the metrics that make the formula for revenue are also universally accessible too. And so these definitions can be accessible in other tools. They can be accessible in these four here, but they can also be accessible in things like data discovery and catalog products. They can be accessible in monitoring tools and all of that. Now what this does is this makes our lives easier. And that's really nice. It represents the sort of standards of the modern data stack, where data and metrics are modular, they're version controlled. It also consolidates governance into a clean layer where governance lives in the entire left hand side of this diagram and consumption lives on this kind of right array. But what it really does, the real impact of this is it means all these tools on the right can stop worrying about data governance and metric consistency. That means these tools on the right can focus on building great last mile experiences for improving whatever they were built to improve. So it means that the right side of this diagram no longer has to worry about what the Rubik's Cube looks like and can instead just focus on solving it. What that means is metrics are the beginning of a platform, they are the beginning of an operating system for the data stack. And in the talk that Tristan and Drew had, Martin talked about how data is going to be a trillion dollar market. And there's a question of how we get there. And this is how we get there. This is how we make everybody a data buyer, where metrics and the things underneath it are platform and the apps on top of it are experiences. And so that's why I think today marks the end of the age of the modern data stack and the beginning of the age of the modern data experience, where we're thinking about building these experiences on top of the stack that we now have. Now, that doesn't mean we're done a long way from it, because as great as this diagram is, which I really like this diagram, blueprints aren't experiences. This map is not the territory. And there's a lot of questions about what exactly the experience of using metrics should be. And so there's a ton of different tools out there. And I know nothing about most of these. I know nothing about monitoring and discovery and operations and AI tools. I do know stuff about these two though. So I want to focus on these two of BI and analytics, which I kind of think of as like general data consumption and exploration. These are the tools you use. When you have a question, you want to be able to answer it, you want to be able to explore data, those kinds of things. This represents only a part of what data is useful for. I recognize that, but it's an important part. And so I want to talk about how we would experience using metrics in this world. And so within these two boxes, two big questions stand out to me. First, how do we interact with metrics? That most implementations of metrics are code oriented. The thing that Drew demoed was was code. A lot of the other tools that are building sorts of things are primarily code oriented. This makes sense. That's how you make it accessible to a bunch of different other tools. But in order to make it consumable for other people, it has to be translated into some interface that probably isn't code. And today, the way we have historically built that, and especially in self serve tools or BI tools, is around the concept of tables and columns. That you see this on the left of tableau, you see these dimensions and measures which are mostly representing tables and columns that are in tableau. You see the same kind of paradigm and looker, you see the same paradigm in mode as well. That most of this is constructed around the notion of tables and columns. Similarly, tables and columns are obviously very important in SQL where that is kind of the primary nouns that we would use. Metrics don't really fit into this paradigm. If you want to look at a metric, you may not care or even know what the underlying table is. When Drew was demoing what he was demoing, the idea of looking up something like revenue, we don't know what table that's underneath. That's not the point. The point is the metric. The noun there is a metric, not a table. And so outside of interfaces within some metrics, there are tools. This is Transform, which is also building a metrics layer. We don't really have a model for understanding how do we interact with data that's built around metrics rather than tables. So we need to figure that out. The second thing we have to be able to do is how do we actually use metrics in different tools? Because while there are only two boxes here, there's actually a lot of different ways that we consume data within even just these two boxes. For instance, we've all got dashboards, we have data exploration tools, we've obviously got SQL tools, we've got notebooks, we've got data apps. All of these things, if you put them all together, are different experiences for consuming data. And so this kind of zoomed in view of those two boxes, we have to be able to figure out how do metrics actually solve problems across all of these experiences? Which raises the third question, which is not only how do we solve problems across these different tools, but how do we, or how do we solve problems in these tools? How do it across the tools? How do we do it? Where you're actually bouncing between different types of ways of interacting with data? Because really what happens if you're using all these things is you're not just using these things in isolation. You're often finding yourself hopping between them, between going from dashboards to notebooks, going from SQL based analysis to a data app to a visual analysis, all those sorts of things. When that happens, we often find ourselves falling into the gaps in between. An example that probably lots of people are familiar with is you've got a dashboard in your BI tool. Something in that dashboard is down. Say signups here is not looking so good. Some angry exec comes along and says, hey, what's going on? You've got to figure that out. And it's actually pretty hard to extend that work straight from the BI tool. In the Looker case, for instance, Looker will give you the SQL that generated that Dashboard, but it's pretty hard to use. It's like, I mean, this is no knock on looker. It's like it's a machine generated code. It's just not really designed for people. And so it's difficult to be able to take this work and extend it easily into trying to figure out what happened with science. And what often happens when we need to do that is we end up kind of starting over, opening up a blank SQL editor and saying, okay, like let's dive in, into this. And so to follow on to what Erica Louis was talking about at the very first talk of this conference, this doesn't really mean we're building more knowledge. We're really just kind of repeating it over and over. And so the question is, can the metrics layer turn this disjointed experience with these gaps between ways of working into something that's more connected, into something where these things are actually more, more seamless? And I think it's worth trying to do this because if it does, there's a really great consumption experience to be built on top of this. And so to me, what that experience could look like is people should be able to start from a library of metrics, choosing the ones they want to look at, be able to choose the metric, choose a variety of details, say things like time, dimensions, grouping, filters, all the things that Drew just showed for how to present that data, create data on top of it, so get results back. In some cases, those tables could be pretty wide. You want to be able to add multiple dimensions, add multiple metrics, derive metrics, then you want to be able to visualize that result. And this is more than simple charts. You want to be able to do it in pivot tables and rich visualizations and great ways to both understand that data and to present it. And then you want to load those metrics into, into visualizations, into different outputs. It may be creating dashboards off those metrics, it may be creating narrative reports, it may be creating data apps, but you want to be able to do all sorts of these things with it. And this also shouldn't just be for business users or people who aren't wanting to write code, that analysts should be able to follow the same workflow as well, either starting from a dashboard and then starting from scratch as well. For the questions they have, they should be able to use metrics as a shortcut to get the first level of answers they need to get. And then when they get to a point where metrics no longer answer that question, they should be able to open up more complex analysis by easily extending the SQL and the queries that are generated by the metrics themselves. So they should be able to wrap those queries with their own SQL, being able to enrich the metric, bringing in new model tables, bringing in new metrics, or even incorporating raw data, if that's what the question calls for. And then if analysts want to be able to go further, they should be able to open all of this up into notebooks or other rich environments for being able to explore that data using the type of deep statistical analysis that analysts often want. And finally, they should be able to also create the same sorts of things. Analysts aren't just building one off reports, they're also building dashboards and things like that for sharing their workflows to facilitate that as well. And if we're able to build that, this experience actually goes one step further, which is rather than just connecting these different ways of working, we can blend them into one seamless experience. Because none of these modes are perfect. Like dashboards aren't perfect, visual analysis isn't perfect. SQL analysis notebook, none of these are the perfect mode for working on any one problem. Sometimes we need code, sometimes we need simple UIs, sometimes we need rich visualization, sometimes we need Python and R, and we need to be able to move between these things depending on what problem we're trying to solve or what step in solving that problem we're actually in. And so to me, this experience also merges BI and data science in a way where data science workflows can use the more powerful governance tools that are traditionally associated with bi. And governance can be extended to answer questions that weren't anticipated, where governance is typically more rigid. And so I think Martin also talked about sort of the collision of these things. This also facilitates that sort of collision. And also critically, this isn't Persona based. So where people live in one tool and some other people live in another, code free exploration workflows should be great shortcuts for analysts, as I said, so long as they can be extended with the code when they need to be. And code rich analysis should be great for non analysts so long as they can interact with the result as they would with a traditional bi. So by having everybody be able to work together in this sort of way, we can turn an analytical experience into a melting pot, into a place for the red people, for the blue people, and of course for all the purple people in between. And to me, that is the modern data experience, that is the next era of data consumption. And so as the analyst who lives in these diagrams. This is the experience that I want to live in. As the pointy haired exec who has to consume data and care about dashboards, matching and things like that, and has a board to report to. This is also the experience that I want to live in. But I am a third thing. I am also a founder of a data startup and so I can also say this is experience I want to build. And truthfully, it's the experience that I've wanted to build at Mode for a long time. And it's an experience that we have been building, including, for instance, we had a big launch of new visualizations just two days ago. If you haven't seen it, check it out. But we wanted to build this and we haven't been able to because in addition to building this outer ring, we've also had to focus a lot on what does the Rubik's Cube look like. We've had to define our own system of tons or of meters or of ways to measure things along with everybody else, and that has prevented us from doing this. So the metrics layer changes all this that we can now build on top of it and focus just on that experience of using metrics as opposed to thinking about how do we govern them. And so over the last little while, we've been talking with folks at DBT about their vision for the metrics layer. And in those conversations, we've been thrilled to see how much it lines up with our vision for where we want to go with things as well. And so we believe with the platform that they're creating, with what Drew just announced, that there are things that we can do on top of that. So before I go, I, like Drew, have one more thing that I want to be able to show you. I want to show you what it is that we've started to build within Mode. You'll soon be able to create Mode reports rather than just starting with a query, also starting with a metric. Then people can choose, say, from a list of defined metrics. In that experience, they can choose the various properties that are governed that metric, things like time, dimensions, dimensions to group by all those kinds of things, they can choose the right filters to apply to that metric and then run it all without writing SQL to be able to get these governed metrics that we've seen before. If they want to be able to visualize it, they can do that in nodes. New Visual Explorer, which as I said, was a big rewrite of our visualizations, check it out and create reports and dashboards on top of that, then if analysts want to follow the same flow, they can. But if they want to extend it, they'll also be able to open this up, see the query underneath it, and see the rendered SQL that generates that metric. And if they want to use this as it is, they can. But also if they want to use these metric queries for building blocks with custom SQL, wrapping that SQL directly around custom code, that'll be possible as well. And then this obviously all works in mode, as it does today, where these queries can be extended with visualizations loaded into a notebook for further analysis and things like that. So we'll have a lot more to say about this in the coming months. In the meantime, if you're interested in learning more, please reach out. You can go to node.com coalesce since us your email also reach out over the DBT slack. We'll be happy to talk about it as we progress. We'd love to get your perspective on this, your feedback on kind of what you'd like to see in your take on whether or not the metrics there actually is the future of analytics. So with that, I'll stop there and hand it back over to Anna.
Wonderful. Thank you so much, Ben. That was really, really awesome. And I'm very excited to see some questions roll in. Um, I have one for you already and that is which is your favorite song? Can you guess off of which album?
Well, there would be two to choose from. Good for you is the obvious answer from the Olivia regal album on 1989. I don't know. You know, there's a lot of good things there. I'm not actually as big of a fan of the. The Taylor's versions of things as. As perhaps I should be or as my Spotify rap would tell me that I am. So maybe I am. Maybe the data is not lying.
Say it isn't. So.
Yeah.
The other question was, do we all get free mode T shirts?
If you want them, reach out. We can probably figure out a way to get some folks some T shirts.
Yes, folks are asking me important questions. I have a question for you.
Yes.
What do you think we're still missing.
Like, in the overall experience?
Yeah. You know, we talked a lot today. We're all very excited. But like, obviously there's more work left to do. What's the work that we have left to do?
So I think that this thing that I've discussed, there's like a. I don't know if you can hear this phone ringing in the background. I know what it is. I Love that you don't know what.
It is.
Building these sorts of experience. I think there is, like, this kind of general consumption experience is important, and it's the way that people should be able to use data if they're trying to answer questions. So if you have a question you want to go to, answer, answer that. As Drew alluded to, there is another way in which we consume data. That is, how do we consume it sort of in the moment that we were actually trying to use it? What is the. What does it look like in Salesforce? What does it look like in a design tool? What does it look like in a support tool? What does it look like in sort of the embedded way? Not like an embedded dashboard, but embedded in. The analogy I've used before is like Yelp, where data is embedded in our decision about which restaurant we go to, how do we make a decision about, or how do we embed data in that way. And I think there needs to be kind of a platform for us to build that on top of. To me, the metrics layer is the beginning of that. Like, it is the beginning of using the data stack as a platform for building those sorts of apps and those sort of, like, to me, that is what a data app actually is like. Yelp is a data app, not a. Not a sort of like drag and drop thing with some filters. But we're getting close to that. And I think what y' all announced today is. Is. Is representing that. That sort of first step. But I do think there is stuff that's missing to actually make it sort of a true platform, to the point where we can. We can all kind of easily build those sorts of apps. That doesn't, I think, mean that they're. They're BI or Analytics or kind of the general consumption goes away. I think it just means that data gets consumed in a lot of different ways. And the kind of general consumption thing is always something that will be there.
You mean BI is not dead? Long live bi.
At this point, I don't know if BI is dead or not. I don't even know what I think. I don't know what the last thing I said about it was. I think it's dead. I think it was dead.
I think it's dead. We'll see. Maybe Mila can pull up a link for us and throw it in the chat. Gregor has a question. Gregor wants to know if this is the first time that there are parameters for SQL Table. Has this syntax been seen before?
I'm sure that exists. I mean, I think There are probably lots of people who built internal versions of this with, with some sort of like ungodly mess of stored procedures and things like that where there are probably stored procedures with functions and stuff in them. That's not. I haven't seen it in something that is so foundational. It's often something again, kind of in the way like metrics get defined in every tool. I'm sure there are lots of companies that define this kind of process in fairly narrow scopes, but I haven't seen kind of like the perennial parameterized SQL query of sorts at the sort of scale that, that it potentially could be happening around this again. I'm sure in all of these things, I'm sure Drew was, he was going back and doing research for his talk. He found like tons of things that were like, oh my God, I can't believe the world worked that way. If you were to go back and do research on the various like ins and outs of every BI tool and things like that. There are so many different languages that people have built that probably solve all these problems. Like we are just repeating solving those same problems over again. So I'm sure there is somewhere we can find that has happened. But, but I don't think I've seen it in sort of a platform sort of way before.
Yeah, yeah. It's kind of like Microsoft got to touchscreens in 2001 long before anyone needed one on a laptop. Here's a hot take question for you. What is the slowest part of the data stack today?
The slowest part? I mean, probably dbt. And it's not, it's not so much like, not like DBT's compilation time. It's slow. It's that DBT does some of the hardest work. DBT takes a giant log of tables and says, hey, transform this into something that is, that is like well governed and narrow and that sort of thing. Like, I don't, that doesn't mean it's necessarily slow, but, but I think there is, there is a certain latency in that. There's a latency in the ETL or the, the ELT step partly because you are, you are bound by what the source can do. Like you can't pull from Salesforce. I mean, I think five trans work on this sort of stuff, but you can't pull from some places all the time because you have like API limits and that sort of thing. So overall it's not that slow. The honestly, probably the answer is the slowest part is like how quickly humans can react to IT and systems where it's, you know, AI stuff and it needs to be, you know, I'm going to say something and I'm sure I'm going to start seeing ads for it in 15 seconds from now. But, but in other systems where you're trying to make decisions like the humans willing to make those, willingness to make those decisions is often lot slower than the data that gets loaded in the database.
One last question, then we're going to wrap up. Blake wants to know if mode is going to integrate with the DBT metrics layer.
We'll have that conversation. Drew said to reach out. We'll reach out.
Okay. Thanks so much, Ben, for all of your time. There's more questions in the chat. There are many more hot takes. I would love your hot takes to those hot takes. And I promised folks a very special hot take from Ben, so you're going to see it. And there are more sessions coming up right after this, so please stay tuned.
Great. Thanks, Anna.
Thank you.
The Analytics Engineering Podcast is sponsored by DBT Labs and is hosted by Tristan Handy and myself, Julia Schottenstein. Have comments, questions or guest suggestions? Email us at podcast at dbtlabs. Com. Our producers are Jeff Fox and David Krevitt. If you enjoy the show, please drop a review or share it with a friend. Thanks for listening.
More from Benn

The Bundling vs Unbundling Debate w/ Tristan, Benn Stancil and David Jayatillake

How AI is Changing Data in a Weird Way (with Benn Stancil, CTO at Mode & Field CTO at ThoughtSpot)

[AI and the Modern Data Stack] #181 Why the Future of AI in Data will be Weird with Benn Stancil

Unveiling the Secrets of Effective Data Collaboration with Mode’s Benn Stancil


