#116 Value Creation Within the Modern Data Stack

Hi, I'm Maggie, VP of Curriculum at datacamp, and I know firsthand how overwhelming it can be to keep up with the rapid changes in data and AI. But staying ahead doesn't have to be hard. That's why our curriculum team has curated the best learning content to help you and your team stay ahead, covering everything from data and AI literacy for busy executives to cutting edge generative AI for developers, and everything in between. Whether you're upskilling yourself or your team, we make learning practical, engaging and flexible. It just works. Power your data and AI transformation journey on datacamp.com.

You're listening to Data Framed, a podcast by Data Camp. In this show, you'll hear all the latest trends and insights in data science. Whether you're just getting started in your data career or you're a data leader looking to scale data driven decisions in your organization, join us for in depth discussion with data and analytics leaders at the forefront of the data revolution. Let's dive right in.

Hello everyone, this is Adele, Data science educator and evangelist at datacamp. Arguably one of the most exciting yet daunting aspects of data today is just how much the tooling space has evolved in the past five years or so. Almost every year now we see the rise of new categories of tools, breakthroughs and value creation with data, new emerging tools and techniques, and much more. A lot of these new tools sit within a relatively well defined architecture, one that is often called the modern data stack. Yet a lot of the time the modern data stack tends to be an illusory term, and data leaders approaching value creation with the modern data stack need to also think about beyond just tooling, but align their data work with business value. And there's no better person to deep dive on these topics with than Yali Sassoon. Yali is the Co Founder and Chief Strategy Officer at Snowplow Analytics. Throughout the episode we spoke about what exactly is the modern data stack, its strengths and pitfalls, how data teams should think about value creation, the limitations of data collection versus data creation, and much more. If you enjoyed this episode, make sure to let us know in the comments. And now, onto today's episode. Yali, it's great to have you on the show.

Adele. Thanks so much for having me. It's an absolute pleasure to be on with you.

Awesome. So I'm excited to speak with you about the modern data stack, data creation and why it's important your work leading Snowplow and more. But before maybe give us a bit of a background about yourself.

I'm a data guy. I'VE been a data guy my whole professional life. I'm quite old, so that's coming up to 20 years now if you go way back. I studied science and maths at university and then ended up doing a master's in history and philosophy of science. I was really kind of interested in how over time we've learned as people to use numbers and kind of numeric reasoning in more and more different fields. So when I left university, I was kind of excited to see how numbers and numeric reasoning could be used in a commercial context. And so I went into consultancy. I worked as a strategy consultant and an operations consultant, really using data to try and help companies do what they did better. And I've worked in data ever since, and particularly at the intersection of data and technology. I guess those are my two passions.

That's really great. I'm very excited to expand upon these passions and something that you definitely have a name for is one leading snowplow, but also really focusing on the modern data stack and your writing and the way you approach data. As someone who's been in this space for more than five years, talking about myself here, I think in a lot of ways one of the more illusory or confusing terms that I've seen in the data space is the modern data stack, because in certain ways it's referred to as a category of tools, as an architecture in itself, and even sometimes as an approach to building data platforms. Well, definitely we have a lot to talk about the modern data stack. I'd love to first start off by really dispelling any confusion around this term and get your definition for what the modern data stack is.

I'll try and define it as clearly as possible. You sort of called out three aspects and I think all of them are sort of wrapped up in that idea of the modern data stack. So first up, sort of a disclaimer. I don't think it's a very well named term. I don't like the phrase modern data stack, but it is what it is and it's widely used, so let's go with it. At the heart of the modern data stack is the idea of a cloud data warehouse or lakehouse. So this architecture, when people are talking about it, and I do think of it first and foremost as an architecture, is an architecture with cloud data warehouse or lakehouse at its core, and then specialized technologies for ingesting data on the left into the cloud data warehouse and transforming, visualizing and activating that data on the right, and then potentially layering across the entire stack other specialized tools to support things like governance and data quality and data accessibility. So it's an architectural pattern in the sense that you've got this particular architectural approach, but then it's also a set of tools in that the architecture is built around these very specific tools in the middle, the warehouse, and these very specific warehouse and lakehouse specific tools on the left and the right. And then there's this idea of a data platform, but I think that's secondary. I think the definition of the modern data stack is primarily about tools and architecture.

That's really great. And definitely you mentioned here kind of centering the modern data stack around architecture and tooling. We've definitely seen a lot of thought, leadership and emerging best practices on what an ideal data stack or architecture looks like. For example, one of the more distinct ones we've seen over the past two years comes from Andreessen and Horowitz, which we'll link in the show notes for the audience. Maybe walk us through a bit in more detail what the layers of the modern data architecture or the modern data stack looks like and kind of whether the tooling within the modern data stack is converging to be more specialized or do you find it as part of a broad tooling layer from one provider, for example.

So in terms of the different parts of the modern data stack architecture, it makes sense, I think, to start with the warehouse, and I'll use warehouse and lakehouse interchangeably because it's frustrating, say one or the other, and the two are really converging. You've got the warehouse in the center, and the warehouse is where the data lives. So it performs this storage function, but it's also where all the computation happens. You're bringing the computation to the data. And these warehouses and lake houses, your snowflakes, your databricks, your bigqueries, your redshifts, these technologies provide organizations with the ability to store and compute on data at unprecedented scale in ways that is much, much more economic than was possible beforehand. You've got them and that sort of specialized set of vendors at the center of the modern data stack. And then on the left, everybody always draws these diagrams in pretty much the same way, with the data flowing left to right. On the left, you've got your different sources of data and specialized tools for bringing that data into the data warehouse. And it's pretty common for those diagrams to distinguish between tools like FiveTran, AirByte, Matillion that extract data that an organization has, and the different tools and applications that are in the business and the different third party SaaS applications, things like your Salesforce and your Zendesk, and then the different databases that might be dotted around your organization, you know, Postgres over here and Mongo over there and replicate that data into the data warehouse on the one hand, and then other vendors and providers to pull in what in these diagrams is typically labeled as events, your web data, your mobile data. Real time data that's generated as you engage with your customers. And that's normally the part of the stack that snowplow. The company I co founded is drawn on is often where you see other companies like Segment and Rudderstack drawn as alternative sources for that sort of data. Then within the data warehouse, data warehouse provides this computation. There's typically work to be done to transform the data that comes in into different derived data sets that can be used by the different teams. And there, I think the two vendors that probably get most attention are DBT and potentially Matillion. And then on the right you've got your different BI tools for visualizing and consuming that data, your lookers and so on. You've got your different reverse ETL vendors that will take the data out of the cloud data warehouses and push it back into your different SaaS, applications or different production systems to actually activate. So if you've built for example, an audience segment of people you think have a high propensity to buy, you can push that into Meta or Google and go and target those users in those segments. And there you've got your census, your high touch reverse ETL vendors, and then across the bottom you've got organizations that will help with things like data quality or data catalog. To think they're things like Amundsen on the data cataloging side or Monte Carlo Great Expectations on the data quality side. So I've given a rough idea of the different boxes and some of the vendors that you'll typically see there. But those are broadly the core different components of the modern data stack. But what we've seen over the last few years is more and more vendors and more and more boxes appear as this kind of ecosystem has exploded around the cloud data warehouse to help organizations execute more and more use cases on that architecture.

And do you think in some sense the ecosystem of tooling here? Because I do remember like a few years ago, for example, the debate oftentimes would be around all in one platform versus best in breed. For example, do you think that we have converged on a best in breed model given the really large amount of new categories of data tools and larger amounts of vendors within these categories.

I don't think it's as simple as that. I think best of breed and the idea of having vendors focused on very specific horizontal slices of the stack has become easier and easier as the modern data stack has evolved, as this ecosystem has developed. But I think organizations should take a sort of a pragmatic approach. So there are certain things that an organization will want to do and they can be done by someone in the line of business with a SaaS tool that's focused on that line of business. And that SAS tool will do the data collection, the processing, provide all the UI and workflow, and then the data activation as a loop. What the modern data stack provides and where companies are sort of building their own data platforms is the opportunity to execute on use cases and build data applications that span across the lines of business or across these different data sources. So when you want to build a customer360, for example, you want to start identifying customers who, like I said, had a high propensity to buy your particular, or maybe a high propensity to churn, or maybe you're looking to build a personalization system to deliver more relevant experiences to your users. When you're starting to do that, you really want to start drawing data on a much broader set of data and you want to start computing on that data in much more intensive ways. And you potentially want to start doing that in something like the modern data stack starts to look really attractive because there you have this rich tool set for doing that computation. So where the computation gets more complicated, more intensive, maybe you're doing AI or advanced analytics, or when you're trying to do things that are more bespoke. So your end to end. SaaS vendors are very focused on core sets of use cases and key companies in them. So if you think about web analytics, if you're a retailer, you're pretty well served by the different web and digital analytics systems that are out there. If you're a two sided marketplace or a subscription business, you've got a model that doesn't necessarily fit, then using this sort of architecture to deliver your basic reporting might make a lot more sense because actually you've got to do a huge amount of work to reconfigure those tools that have been built for one sort of company for another. So this kind of architecture gives you a lot of flexibility for more intensive and more flexible data processing.

Okay, that's really great insight and I really appreciate here how you expanded on the debate with kind of what are the definitions maybe of the tools within the modern data stack. It seems to me that what constitutes a tool within the modern data stack has to do a lot with its ability to scale and really handle a lot of high complexity computations. Would you say that is a defining characteristic of the tool set within the modern data stack?

It really varies by the different boxes. If you think on the ingestion side, for example, the rise of the cloud data warehouses created this opportunity for the five trans and airbytes of the world to provide solutions that replicated data from different places into a single location. And those tools could relentlessly focus on just extracting the data and loading the data and make it somebody else's problem to take the data as it lands, because it lands in a format that's not really useful for anyone out of the box and go and transform it into something useful. So sort of rely on the storage and compute that the cloud data warehouse provided, and then other vendors like DBT to provide the tooling to take that data and fix it up. So there you've almost got. The emergence of this architecture has enabled for increased simplicity rather than complexity in the ingestion space, because you're moving the complexity of the transformation from ingest into the warehouse. And people talk about the switch from ETL to elt that really saved those vendors a lot of work. On the other side, if you think about the tooling for sort of building AI and machine learning models, those tools really became a lot more powerful and a lot more complicated, if you like, because they were able to leverage the power of things like cloud data warehouses and cloud data lakes to sometimes run some really quite complicated machine learning models that otherwise might have been prohibitively expensive to use in production. So really the impact has been quite uneven. I think that there are common characteristics across the different tools in the modern data stack. You've really got to look box by box and think how that function has been transformed by this big old data warehouse that's in the center of this thing.

That's really great. I love how you draw the distinction here, maybe on the insights layer of the modern data stack, because a lot of the audience I would assume here are data scientists, data analysts who work exclusively to a certain extent on the insights layer of the modern data stack. What do you think are kind of distinctions within that category of the tool set that we see today, rather than the tools that, you know, data teams used to use five years ago, for example?

So I think the big difference is the tools in that space that bring the computation into the modern data Stack so they lean heavily on the cloud data warehouses and data lakehouses to do the computation. And for the end users, for your data analysts, for your data scientists, the big benefit that should bring is that they should be able to work with the entire data set directly. They're not in this world of having to download an extract of the data and work on that on their local machines. This should mean that they have a lot more flexibility to work with the data. And it should mean that they can work on the data, the production data, so the warehouse is powerful enough that they can go and they can train models on it and test those models. And then if they're going to put those models live, they're going live on the same data set, ideally. And so some of that workflow around going from development to production should be a lot easier. And some of those concerns that what worked at a small scale might not scale up should be alleviated by working with this sort of architecture and with the tooling that takes advantage of this architecture.

You know, we've talked about in a lot of ways, kind of what are the strengths of the modern data stack. But I know that this is something that I also focus on in a lot of your writing. Given your vantage point, as someone who speaks with a lot of data teams consistently about how they're leveraging the modern data stack, what do you think are some of the current pitfalls or weaknesses of the modern data stack today?

So I think the biggest pitfall, and I don't think it's specific to the modern data stack, but it's definitely worse for the modern data stack than potentially other architectures that the data teams might potentially be adopting, is I see data teams over focus on tools and architecture at the expense of focusing on the lines of business that the data platform needs to serve and delivering real business value to those lines of business. Actually, that last mile piece of making sure that the data has both made a difference, that the business really understands the difference that it's made. So because of the modern data stack is sort of use case agnostic. It's like, hey, look, this is a big architecture and this is going to work for all your use cases. And that's already not necessarily true. Every use case is different, and we should always be open to optimizing the technology for the use case rather than hacking the use case to work with the technologies and the architecture that we want to adopt. But I do see data teams over focusing on the tools and technology. And then there's this kind of playbook where you go and you buy your cloud data warehouse, you go and you buy your ingestion tools, and you go and you bring in all your data. You go through the business, you're like, what? You almost create an inventory of all the different data in the business and you bring it in and then you put a BI tool on the top and maybe put a reverse ETL tool in there, maybe buy some machine learning tool. You're like, okay, great, now I've got a nice, a nice stack. But actually, at that point, you've spent a lot of money. You haven't really delivered any value. You really need to go and use that. And the possibilities to use that, given you've got all the organization's data in there, are kind of limitless. But that sometimes creates an analysis paralysis, or sometimes data teams will spend the time using that to create data sets and then democratize them, publish them, and hope that people come and do useful things with them. But they might more often than I'd like, and I think we in data would like to admit they stop short before the data is actually made the difference. And that is a problem. And the reason is it's exacerbated. There's this sort of law in software development that your technical architecture reflects how you're organized as a company. With the modern data stack, I think there's almost the inverse happening, where data teams will organize around the modern data stack. And I've seen organizations where you have teams that are dedicated to bringing data into the data warehouse and separate teams dedicated to using that data to drive value. And that is a terrible way of organizing because you've totally separated, you've totally decoupled the people who are responsible for bringing the data with the people who are responsible for using the data. And those are two sides of the same coin. There shouldn't be anybody bringing any data in without any idea about how that data is being used. What data you're bringing in should be driven entirely by how you want to use data to drive value. And that's a very extreme sort of failure mode where the technology is coming before the business value. But I've seen that organizational structure adopted a lot, and that's, I think, always a mistake.

Okay, there's definitely a lot to unpack in that answer. This is something on, like, value creation with data and how teams can avoid the pitfalls of shiny toys in a lot of ways is something that we deeply care about on Dataframe. So maybe I'm going to take a bit of a tangent in our conversation and focus on this. You mentioned here that teams can be very invested in the tooling side of things and in ensuring that they have a very holistic data stack. But that doesn't necessarily translate towards creating business value with data and ensuring that different lines of business benefit from that data, which you aptly here kind of captured in that last mile of analytics. Kind of walk us through what is an antidote to that problem in your opinion. How do you approach as a data leader when you know, building out your data team, building out your data stack to have a value driven first approach?

I think the key thing is to have a value driven approach. You start on the right of the stack and you work back to the left. So you start with what are the different ways we can use data to drive value in my business? And then you work back to what is the data we need and how do we use that data to drive that value. And so you focus narrowly on that until you've delivered that value to that end user, to that line of business. And not just that, but you go the extra mile and make sure that the line of business understand the value and the broader business understand the value. And then you go to the next line of business and you pick your next use case, your next data application and you work back. And so the key thing is to start on the right, to start with the value and work back to what's the minimum I need to do, what's the minimum technology I need to put in place, what's the minimum data I need to create to deliver on that value? And then you can iterate, you can go back and you can bring in more data and you can take more sophisticated approaches to computing on that data to drive that value. But you're relentlessly focused on the value and what you need to do upstream with data to deliver that value. And the failure mode is organizations doing it the other way around. They start with the data like, hey, I've got this data set over in Salesforce, so I'm going to go and I'm going to extract it and put it in my cloud data warehouse. That's cool. Okay. And now I'm going to take DBT and I'm going to model that data into a nice set of tables. Okay, that's cool. And now I'm going to build some reports on it. Okay, that's cool. But actually, hang on, I've just spent a bunch of time and effort recreating reports that I had in sales for force anyway, but in a different stack, no value at that point has been delivered, you need to start on the right and work left rather than starting on the left and working right.

So taking that Salesforce example, you know, a good approach here would be actually talking to, you know, the CRO or the Rev Ops team, whoever it is, about what are the limitations that they have with Salesforce right now and how can you build out a reporting suite that actually suits their use cases from that use case that they mentioned? Would that be a correct approach in this situation?

That would be absolutely a correct approach. So you look for those opportunities to do something that you can't do in Salesforce. If that's something on a reporting perspective, maybe there's some view of your customers that requires data in Salesforce, but also data from Zendesk from a support system. You want to understand who are those customers who are engaging with your support desk and maybe at a particular tier or were part of a particular cohort. And so you're combining the data from those two sources and that you've decided that that insight is going to unlock some value that isn't something that you'd be able to use in either of those tools individually. And then the strength of the modern data stack is you can bring that data in, you can combine it, you can deliver that report, you can work with whoever wants to understand that particular subsegment. Maybe you can go and activate that subsegment and run effective marketing campaigns to go and drive value from it, and that's the right way of doing it.

You mentioned here also previously in one of your answers on the pitfalls of value creation with data is that data teams will organize around the technical architecture. That's necessarily not the best way to approach how to build a data team. What do you think is the optimal way then to organize a data team and around which principles necessarily would you tie it to?

So you want to organize a data team around value creation. And so I think the most promising approach is to treat data as a product and bring sort of product management techniques into how you think about managing and socializing data. So you have data product managers who are responsible for understanding the needs of the different people in the business that want data and how they want to use that data to drive value. And then the data product manager is responsible for delivering those data products into the hands of those people to go and help them execute on that value. And they're leading cross functional teams in the same way as a product manager in software development will lead a cross functional team that will include everybody required to deliver some value to an end customer, your front end engineers, your back end engineers, maybe a data person, a designer and so on. Your data product managers working with cross functional group of people, maybe front end engineers who can integrate tracking analytics, engineers who know how to build robust pipelines, data scientists. If the use of the data requires machine learning techniques, data modeling techniques to go and make sure that value is delivered to those different lines of business.

Okay, that's really great. And I think this also marks a great segue to discuss, you know, one of the areas of the modern data stack that you're very invested in, given your position at Snowplow, which is really early in the flow where organizations collect and extract data. You know, I've seen you speak and write about this before, but walk us through maybe in more detail how you think organizations should be thinking about data collection and maybe outline the main limitations of the current way data teams approach data collection.

So this sort of follows pretty directly from the discussion we've just been having, but it really crystallises when you look at how organizations think about the left side of the modern data stack. So sort of ingesting their data. So I think there's a common failure mode where organizations start with what's the data we have? And let's go and bring all that data into the data warehouse and then we'll have all the data that we need and we go figure out how to use that data to drive value. So they're starting on the left and they're working right and they're going, they're creating that inventory of data and they're bringing that all in. And when they're doing that, they're making this assumption that the data that the organization has is all the data that the organization can expect to have. And the best thing they can do is get it all in. And once they're in, that's it, all the data is there and they can go and shift their focus, right to actually taking that data and doing stuff with it. So probably won't surprise you to hear, I think they need to start on the right with the value. They start with the team. Maybe they've got a team, they're at a media company and they want to personalise more of the experience and use personalization to drive up engagement of people with content. And so what that team should be thinking about, well, if we want to build a personalization engine, what's the best data that we could use to feed that with? What are the features we want to use to drive our Engine. And rather than start with what data an organization has, the data scientist data product should be figuring out what data do they want, taking that to, ideally a data product manager and saying, hey, these are the features that we want. Do we have data available to engineer this? And the organization should then go and create the data that the organization needs to deliver that value. And it might be that some of that data is readily available in some source systems and can be extracted, but it equally might be that some data needs to be created from scratch. And actually the organization has the opportunity to go and create that data. But today I don't think most data scientists even think they have permission to say, hey, can we go and create this data that we don't have? They're like, oh, we don't have it. Okay, well, I'll stick to what we've got. And I think that's a huge missed opportunity in a failure mode.

Yeah, and I really want to deep dive into that of like, what data creation or what successful data creation looks like. Oftentimes when we hear about the term data creation or leveraging data that the organization necessarily has not collected, it's usually constrained to the area of synthetic data generation. But I think something that you mean here as well is that being able to get data on your customers, for example, and this hypothetical situation where you're a media company that you're able to create new features from on your particular customers as well, not just synthetic data. Can you walk us through maybe the distinction from synthetic data with what you mean here by data creation? And secondly, walk through maybe an example of what successful data creation looks like?

Great question. And I definitely don't mean synthetic data over here. So let's go into this in a bit of depth. When we build reports or when we build models, because we want to understand the behavior of a person, we want to predict the behavior of a person or a machine, and we want to influence the behavior of a person or machine. So we might want to influence a person to engage more in our product or to buy new products. We might want to make a prediction about a machine, maybe to do some proactive maintenance. So we're in this world of wanting to use data to better understand things, people or systems, and make predictions about them. And so the data that we need to create is nearly always about those people or machines. And so we tend to call this behavioral data. It's a term that's reasonably well used. But I think we define behavioural data pretty widely to mean data that describes what people or machines are doing second by second and minute by minute and the context in which that behaviour is taking place. I'd call all of that behavioral data. And that's really valuable data because it's very, very explanatory. You can see if it's data about a person, you can see the decisions that that individual has made and the context that they've made those decisions and how they've gone about making those decisions. That's very valuable data if you want to understand what decisions they're likely to make or influence those decisions, understand how context and environment impact the way they make decisions. And it's the same for machines. Machines aren't. Well, sometimes machines are making decisions, but sometimes you're just trying to understand what drives the behavior of those machines. And we have the opportunity to create data that describes those people or describes those machines. So with Snowplow, we've built a platform to do just that. So the idea is that there are SDKs that you'd integrate into your different applications, where either your customers or your employees, whoever the people you are that you're engaging with, you'd integrate the SDKs in the applications. That engagement is happening and those SDKs would create data as that behavior is happening in real time. So we provide open source software to do that. But there are lots of other solutions out there. Fundamentally, there's behavior that's going on from people or machines. There are systems where that behavior is happening. There might be websites, applications, backend systems, and there's the opportunity to update those applications with things like SDKs to make it easy to create the data as those things are happening that describes that behavior and emit that data as events. And so we can create that data in real time. We can optimize exactly what data we're creating, what fields we're creating, how we're structuring that data to make our lives as easy as possible, to take that data and then go and execute that use case, to go and create that value that we want downstream. So if we're creating our personalization engine and we decide that it's really important to understand the amount of time that people have spent engaging with different content items, because it's something we've seen across our user base and customer base's time spent is very predictive of a whole raft of things, then you make sure you're tracking the events so that you can derive how long somebody spent. You track when someone loads an article, maybe track a heartbeat. So as long as they're engaging with that article, you're keeping tabs. And as soon as they background that web page, you note that and you note that they're no longer engaging with the article and they come back and you start tracking that again. You're emitting that data in a way that makes it very easy to take that data and aggregate that data and say, hey, this person engaged with this item of content for 13 seconds and this item of content for 2 minutes and 15 seconds. And then you can go and feed that into your model and make your prediction as your example. So you're driven by the feature that you want to create to make your prediction, and you're creating the data in a way that makes it as easy as possible to engineer that feature.

That's really great. The first thing that pops to my mind here when you're talking about this behavioral data would be something like a customer churn model. Right. Oftentimes organizations build customer churn models by just understanding, okay, what is the package, what is the lifetime value? Or, like, the different products that a customer has bought from us? And they use these as like, features that are likely predictors of someone to churn. But if. What if you're also able to go much deeper than that and see how someone exactly is using your application in comparison versus, you know, the rest of the user audience, what time of day they're looking at certain things, how deeply engaged they are. Would you think that that's kind of an illustrative example of how behavioral data can elevate the performance of a predictive model here?

That is an excellent example. So, yeah, to take your example of a churn model, when you're coming up with the first version of the model, you might have some hypotheses, like, if the person's been around my account page, there might be a set of bad signs that mean they're likely to churn. There might be a set of behaviors that mean they're really engaged, they're likely to stay engaged. And you build your first version of the model model, and hopefully that's reasonably simple. You've got a small set of really predictive features. And then over time, you might add more and more features like, well, I wonder if this is going to make the prediction more accurate. I wonder if this is going to make the prediction more accurate. And one of the benefits of data creation is you can go back and you can keep adding more and more data to engineer more and more features to make your model better and better. And at each point, you can test your new model versus your old model. You can say, hey, look, we've gone and we've had to create all this extra data that has this extra cost implication. There's more computation that's required and our model's this much more effective. What's the return on that investment? Is it worth investing more and trying to drive better improvement in the performance of the model? Or have we now plateaued and we should put our effort somewhere else? So this data creation means that as a data scientist, you can evolve your approach and you can keep going deeper and deeper if that makes sense for you and the business, and build better and better models by creating better and better data sets and richer and richer feature sets to feed the models.

That's awesome. And you know, one kind of truism in data science a lot of ways is that, you know, Data cleaning is 80% of the data scientists workflow or the data team's workflow. What are kind of, you know, data quality considerations or nuances that are applied to data creation or when creating data rather than collecting data? Is there any difference in the data quality kind of checklist a data team needs to have? Is there benefits for data quality given that you're creating this data in house that you don't get to have? Maybe from collecting data? Walk us through. Maybe just the data quality nuances of data collection versus data creation.

Yeah. So when you're creating the data, you at a huge advantage when it comes to data quality versus data extraction. So this truism about data scientists spending all their time cleaning their data and no real time actually using the data to engineer the features and drive model performance, that's because in the vast majority of businesses and the vast majority of use cases, the data scientist is stuck with extracted data. So fundamentally their starting point is data that hasn't been deliberately created for machine learning. And so there is no assurance around the accuracy or the completeness of that data. The data scientist has got to go and figure out, can I trust this stuff? Do I believe it? Does it look accurate? Does it look complete? Then the structure of that data has been optimized for whatever application that data came out of. It's not been optimized to feed the model. So the board data scientist has got to go off and transform data that was created for one purpose and try and transform it into something that's suitable for another. And so there's a huge amount of work that's required there. And then if the model's successful, you need to scale that to run in production. And that's another load of work. And data quality has become, rightly, this very, very big issue. And it's become this big issue over the last few years, really, because organizations are trying to use data in more and more sophisticated ways, and AI has been the primary driver of that. And the more sophisticated your use of data, the higher the quality bar has to be. So, to go back to digital analytics and a web analytics example, if you're using your web data to tell you how many visitors you have every day, it really doesn't matter all that much if 5 or 10% of your data is rubbish, because you can still see the trend over time, and you can still use that data to understand the impact of marketing campaigns in driving traffic levels up, or if there's a dip in traffic, this sort of inaccuracy isn't going to matter. But if you want to start using that same data to personalise an experience, a few lines of missing data can lead to you making a completely bogus prediction, a completely misunderstand who this user is and what they're interested in. So the more sophisticated approaches to data require higher levels of data quality, but the data that the data scientist is stuck with is extracted data. It's data that doesn't have that. And so you have this plethora of really wonderful solutions in the data quality space, there's lots of really smart people working on data quality. You know, the folks at Great Expectations, the folks at Monte Carlo and so on. But they're fundamentally building tooling to help build assurance that if you've got good data in, you're not introducing extra data quality issues at each stage of the data processing. So they're building assurance in your data pipelines and the integrity of the data pipeline process and the integrity of the data at each stage in that process. Unfortunately, the vast majority of data quality issues happen at source. So in digital data, the biggest challenges are somebody's put an application developer's made some change to a website, some benign change, without thinking about data, and that's accidentally broken the tracking or changed the tracking, and you can't fix that after the fact. That's a problem at source. And lots of data quality issues stem from source. If you think about Salesforce data, you know, as people not entering the right data in the field, are people not bothering to fill in the right forms, and none of these data quality issues can be addressed after the fact. If you're creating the data, you can make sure that you're creating it systematically. You're building those checks into the data at the point of creation. You're writing those unit tests that are simulating your customers. If you're creating data about them or your machines making decisions, performing different actions, and making sure that the data reflects that exactly. So you're catching those issues right at the beginning. And so you're designing the data fit for purpose right from the beginning. And so you don't have some of those data quality challenges, and you don't have those data transformation challenges as well, because you're landing the data in a format that's much, much easier to pick up and use.

That's great nuance and I love how you still contextualize a lot of the data quality issues data teams face with it being at the source data. So even if you do have something like a Monte Carlo or gray expectations, the adage of garbage in, garbage out will remain the same if data is garbage at the source level.

That's exactly right.

So given that you've seen a lot of teams work with creating data effectively, or have seen a lot of data teams create data with Snowplow, what have you seen to be some of the main best practices organizations adopt when creating data effectively and how has this impacted their data products and outputs downstream?

So I think there's a kind of journey we've seen organizations go on, the ones that have been really successful with data creation. So they'll typically start with some sort of centralized team who are looking at driving data literacy and impact of data and business up. So typically a data team, and they'll be focused on one or two high value use cases that will really make a difference to the business. And the business hasn't been able to execute and they'll partner with the relevant line of business to execute on that use case. And then what they'll do is they'll work to socialize that success with the other lines of business and slowly work with more and more of the different lines of business to create data and use it to drive more value. And so you're slowly and sequentially building up a richer and richer data set as you're creating more and more data for more and more different use cases. And then the key thing is that there's this inflection point where the different lines of business start to feel important, empowered. Okay, I get this. I understand now that we have this ability to go and create data and use that to power new machine learning models and use that to deliver new services or to build new reports and socialize that with our customers to drive more value or whatever it is. And at that point they start taking the reins and you start seeing a much more decentralized approach to really creating and using data. And at that point, the central data team become a team of facilitators and governance experts. They sort of support the other teams, but let them drive it. They push for standards sometimes to stop kind of data sprawl and too much data proliferation. They maybe work with the compliance team to make sure that the creation and the use of data is all compliant. But you move from this kind of decentralized to this decentralized model where the different people that are using the data actually driving that data creation process. And we're seeing that in more and more organizations now. It typically takes two to three years to work through that cycle. But every time we see that inflection point, we're like, okay, this is a customer that's really running now and really using data creation to the full.

Yeah, we can definitely dedicate an entire episode to chatting about this. But I think one thing that definitely cuts across a lot of guests that come on Data Frame to talk about, you know, successes of data analytics leaders, whether it's in data creation, value creation with data, et cetera, is, you know, the importance of evangelism and kind of creating that success story narrative within the organization, which is very essential. So in a lot of ways, what we've touched on throughout the episode is really around the challenges and opportunities of leading data teams and successful data teams in 2022. You know, we touched upon a more technical topic today, but there's definitely a lot of room here, as we mentioned, in kind of evangelism, for example, a cultural aspect as well of succeeding as a data leader. The data space is shifting rapidly and so are the skills and the principles required to lead data teams today effectively. Maybe walk us through leadership principles, whether technical, operational, cultural principles you think data leaders should be integrating in their day to day to day, especially when it comes to making or creating value out of the modern data stack.

It's a great question. I'm going to sound a bit like a broken record by going back to the cultural piece and the starting with value. Let's the macroeconomic environment we're in now, I think companies and CFOs are very focused on cutting costs. And if I'm spending all this money on X, what's the return? What's the justification? Should we carry on spending it? And I think we in data have enjoyed many years of being able data leaders have been able to go and buy tools and build teams because everybody believes in the mantra of data is good and data sophistication drives value. And I fully believe that use right data generates enormous amounts of value. That's really the thing that really excites me about data, its enormous transformative potential. But data leaders need now to be that much better at doing the last mile, that much better at getting to value, and that much better at evangelizing the value than they ever were before. Because I think that the spotlight's really going to be on them and those budgets have really crept up. The modern data stack is great, but it's not cheap. If you're bringing in large volumes of data, if you're doing a lot of computation, your snowflake and your databricks bills are going to rack right up. Now, if that's all driving value, and it should be, you should be seeing 100x return on that investment or more. But there are definitely data teams that are spending that money that aren't seeing that investment, not because they don't have the technical skills, or not because they're not following the right architectural patterns or trying to solve the right problems, but because they're not necessarily engaging with the lines of business in the right way. They're either not getting those lines of business to value, or they're not doing the evangelism so that value is recognized and then those expenses are at risk. I'd urge all data leaders to spend as much time as they can being outwardly focused on the rest of the business and coaching their data teams to do the same. And as far as possible, to try and organize those centralized data teams around the needs of the broader business rather than the technology that they're integrating. So don't make the mistake that we talked about right at the start of the show of dividing your team up by the different parts of the modern data stack. You want those cross functional teams oriented on helping the different lines of business and being able to bring together everybody who's required to get that line of business to value. And that I think is going to be the key thing that differentiates those data leaders that are successful next year and the year after with those that aren't.

That's really great. So as we close down the episode, you know, one thing I'd be remiss not to talk to you about, especially given that we're ending the end of Q4, is what are your predictions for the modern data stack for 2023 and kind of what are the trends that you're excited about?

So if I think about the modern Data stack. I think we're going to continue to see a huge amount of innovation around in the stack. I think some of the most exciting developments all the way through the stack, I think we're seeing a lot of innovation from the cloud, data warehouses and lakehouses themselves, especially around real time and enabling people to do more real time computation, which is particularly exciting because we're starting to see this stack being used not just for reporting and analytics, but actually to drive production systems. And so the folks at Snowflake and the folks at Databricks are building great new features to make executing and building those data apps on these stacks a lot, lot easier. And then we're seeing the vendors around that respond appropriately. So I think we're going to continue to see innovation in the stack. We're definitely not going to see maybe as many new logos popping up around the modern data stack as we've seen in the last couple of years because the VC5 funding environment has changed quite significantly. The era of sort of cheap money is over. But I definitely don't think there's going to be a slowdown in the level of innovation, which is great. I think you're going to see a lot more focus on how people organize around the modern data stack. I think we're going to see more companies think about data as a product and organize around data products. We're going to see more interest in related concepts, things like data creation and things like data contracts. And I think we're going to have to see more focus on measuring the return of investment in data, which is going to be interesting.

It's an exciting year ahead for data and technology in general. Now, Yali, before we wrap up today's episode, any final call to action for our audience?

Final call to action is I want to bring it back to data creation. Think about the data that you want and don't think that you can't have it. If you think that data that you don't have today is going to make a difference to the data application that you're building. Identify the right set of stakeholders in your organization to go and create it because it's possible and it works and it drives enormous value.

That's really great. Thank you so much, Yali, for coming on Dataframe.

Thank you. Adele.

You've been listening to Dataframe Today, a podcast by datacamp. Keep connected with us by subscribing to the show in your favorite podcast player. Please give us a rating, leave a comment and share episodes you love that helps us keep delivering insights into all things data. Thanks for listening. Until next time.