#128 Unlocking Scalable ROI for Data Teams (with Shane Murray)

You're listening to Data Framed, a podcast by Data Camp. In this show, you'll hear all the latest trends and insights in data science. Whether you're just getting started in your data career or you're a data leader looking to scale data driven decisions in your organization, join us for in depth discussions with data and analytics leaders at the forefront of the data revolution. Let's dive right, right in. Hello everyone, this is Adele, Data science educator and evangelist at datacamp. Arguably one of the biggest challenges for data leaders and data teams in 2023 is the ROI challenge. We're seemingly at the cusp or in the middle of a recession. Boardrooms are looking to weather the storm. They want to see higher efficiencies and more ROI from data teams. And data leaders, especially today, need to develop frameworks and mental models to ensure high ROI from their projects. Enter Shane Murray, Field, CTO at Monte Carlo and former Senior Vice President of Data and Insights at the New York Times. Shane has extensive experience in defining data strategy, building data platforms and teams that deliver roi, and has always cared about delivering impact with data. Throughout the episode we spoke about a myriad of different topics from how data teams can organize themselves around delivering roi, mental frameworks data leaders can adopt for prioritizing high ROI projects, the importance of self service analytics and building a data culture, how to balance short term business priorities with longer term research projects, the state of data quality today, and his work at Monte Carlo, and much more. If you enjoyed this episode, make sure to let us know by liking the episode and subscribing to the show. And now on to Shane Murray. Shane, great to have you on the show.

Thanks Fidel. Nice to be here.

Awesome. So I'm excited to talk to you about how data teams can scale the ROI they deliver in 2023, how to best structure a data team, how data leaders can play a role in breaking down data silos, and more. But what before? Maybe give us a bit of a background about yourself and what led you to where you are today.

Sure, I actually started my career in Sydney, Australia as a statistician doing a lot of work, actually running experiments and I was involved in building and working on one of the early multivariate experimentation platforms in the mid-2000s. That platform, it was called Mometrix. So the company was Mometrics, was actually acquired by accenture in about 2008, at which point I actually moved over to New York where I'm based now. And at Accenture I was focused on taking that software offering and building out experimentation programs in Fortune 500 companies and also helping those companies build out their own teams internally to run experimentation at scale. So we were, I would say, early pioneers of online experimentation using experimental design techniques and choice modeling techniques at the time. And then in 2013, I actually took a role at the New York Times. I spent nine years at the New York Times. So I started as the head of a data analytics team that was pretty newly formed and around 12 people pulled from different parts of the company into this central analytics team that was paired with a research group. And then over my nine years there, I built up that team, eventually merged with a data platform group and a data science group and grew that team to 150 when I was leading data for the Times, the data platform. So where we managed all of our infrastructure, all of the kind of core data products for the company, as well as the kind of user applications and machine learning applications, and then also overseeing all of the analytics and data science teams that were the main users of that. And then so I left the Times. As I said, after nine years, I left in 2021. Took a little bit of a break between roles and then I joined Monte Carlo in this role of field cto where I'm partnering with customers to really roll out data observability at scale and accelerate their data strategies. So whether that's customer facing data products or whether it's rolling out some sort of data mesh implementation, I'm really trying to partner with those executives to drive that in their business.

That's really great and it's a very exciting resume. You know, there's really a lot of things that we can discuss today. One of them is being data observability, your role at Monte Carlo. But where I really want to get at today is how data teams, given your wide ranging experience of delivering high value data teams, is how data teams today can really focus on high value and high ROI projects. You know, we're seemingly at the cusp or in the middle of a recession. Boardrooms are looking to weather the storm. They want to see higher efficiencies and more ROI from data teams. Maybe walk us through from your perspective, what makes a high ROI data function?

Yeah, it's definitely been top of mind in conversations I've had with data leaders over the past six months. And so I would approach this, I guess from three angles of, I think what makes a high ROI data group. The first being, and this may be the most obvious but timely and persuasive, I'd say decision support really achieved through running plenty of Experiments and impactful experiments, often sort of building models to help decision support. Like media, mixed modeling is one form, but various forms of modeling to enable decision support and then obviously reporting and dashboards in general. This area requires good proximity to the business to be timely and relevant. In some cases I've found separation is healthy there just to sort of maintain this objectivity and separation from the group you're working with. But generally proximity is good. The second I'd say which is probably the biggest shift I've seen is this shift from data teams being a cost center and a pure enabler into directly driving revenue. And I've talked about this a bit, but I saw this shift at the times and drove this shift to actually being a revenue driving data team through recommendations, targeting algorithms and customer facing data products. And I think most teams are somewhere on this journey into this shift where they're actually rolling out algorithms or other products into production and they're responsible for really driving revenue for the business. And so I think that's been a critical shift in in ROI for data teams over the last few years. And then I'd say the third component is thinking about that like efficient and extensible data platform. By that I mean firstly, once you invest in a data platform, you should be measuring and ensuring that that platform increases in efficiency over time. As you add new data products, as you onboard more users, more potentially more acquisitions that your company's making, that platform should become much higher ROI as you progress. And then secondly, I use the word extensible. I think it's critical for platform teams to be building ahead of the current needs of the business. I think often you'll find you're responding to the immediate fires of the business or the high priority initiatives of the day. But actually a platform team needs to be be building ahead of the current needs so that once you make an acquisition or build a new product or create a new team, that team can instantly get to value through your data platform.

That's a really appreciate the holistic answer and there's definitely a lot here to unpack. So you mentioned here, first of all, proximity to the business, secondly, making sure that data teams are also delivering like revenue driving data products, not just enabling data products. And thirdly here, making sure that the platform that you're building is efficient, extensible and is ready to drive value in the future. So I want to first start unpacking that first element. I think there are many ways by which we can approach how data leaders can ensure their teams are being productive. Let's start off with that first component. One pitfall we've seen especially early on when data science is often teams are not really close to the business teams that they serve. They're seen more of kind of a support center of a consulting center. Walk us through that pitfall. And how do you think data leaders should be thinking about how to structure their teams for maximum efficiency and value creation when it comes to proximity to the business?

Yeah, it's a really good point and I would say I've seen the pitfalls probably at both ends of the spectrum, but certainly when teams start initially investing in data science and they bring in often academically oriented data scientists into a centralized team, there can be this distance from the business that makes it really difficult to put algorithms into production, in large part because you are lacking proximity from the problem and some sort of realism around how the business works and how it needs to work. I've actually then also seen the other end of the spectrum where teams switch to deeply embedding individual data scientists who may locally optimize a problem and be very appreciated and valuable within the context of a team, but actually not do as much as they could be doing to produce kind of long lasting and sustained value, especially if they leave the company and you have to restart that whole process again. And so I think the most successful models I've seen of this start with a strong centralized team that's both supporting analytics and data science for the business, but also building out that centralized platform as well as this kind of center of excellence model that can enable enable future embedded teams. And so I think the first thing is you don't really start with decentralization. It's always harder to rein in. And when I talk to a lot of data leaders, one of the biggest lessons is don't decentralize too early, even though the demands are often there to enable speed and autonomy and move to these cross functional team structures. And so I've come to the realization that there's a few rules around when you embed or when you decentralize for that efficiency and value creation. And the first one I'd say is the data platform needs to be sufficiently mature so you need to have the tools and processes and standards that enable productive analytical work and similarly data science and machine learning work. Secondly, I think your data team needs to have a sufficient bench so that you're actually embedding managers or directors as well as the IC's individual contributors, so that you can manage in this distributed model while continuing to build a stronger core and center of excellence. And then thirdly, I would say is the business domain you embed with needs to have these clear and distinct objectives. Otherwise you find your data teams are producing overlapping work. And so I think it's critical that the structure that you're embedding in is actually ready for this sort of embedded analytics work. Otherwise you're actually better to stay central and partner with those teams in a slightly separated way. And it is worth remembering throughout that that data scientists and analysts need to work as teams. So embedding individuals has its limitations. And I just add an example there. You know, when I was at the Times, we had data scientists that were embedded in these teams and actually brought them back centrally into the platform to build teams that were led by data scientists that were tackling high value business problems and established a really clear partnership with these objective driven, what we called missions. Right. So they had a senior partner on the business side, whether it was a group growth leader or an engagement leader or a newsroom leader, the data scientists were partnering with on these high value problems and then had the team that could both deliver on that problem, but also start thinking through the shared technology that might support us extending into new use cases and scaling the platform.

That's really great and I love how you outlined the pitfalls here of both of these models. I echo your sentiment here around the decentralized model is that oftentimes data scientists, when they this model is not done right, data scientists will be reporting to a functional leader, for example. And this really hurts the retention of really talented data scientists because there's no really formal codified practices of how data science is done within the organization. Now you mentioned here as well is that you often advise organizations to better start off as centralized in the beginning, build up these capabilities over time. However, there's a trade off that data leaders here face. There is a trade off of delivering short term valuable projects for your partners while also focusing on the long term. So that data science is valuable in the long term and that you're able to build out these capabilities to become a more decentralized function. How do you ensure that data leaders are always managing the trade off between short term projects and long term projects? What are the type of projects that they need to work on consistently so they are aligned with the business value while keeping in mind that fraction of work that is enabling them to build for the longer term?

I think that's key that what you say there about the maintaining that fraction of work, I think as you embed teams, you're essentially starting to then drive the goals of that business unit rather than your own centralized goals. But you also do need to identify key data product opportunities to build or implement that support broader initiatives that maybe aren't requested by your product or marketing or finance partners. And I guess as a leader of data teams, I found that in part you're having to surrender the control to those business leaders to set priorities and then make sure they're held accountable to effectively using the resources. I haven't used chargeback models, but I've heard some teams in this structure effectively use chargeback models to put that accountability in place. But what I found is that it's often a people thing. You need the data leaders within those teams that are capable of prioritizing the thing that no one is asking for and selling that into them, as opposed to this sort of purely the support to the work, I should say to support a product launch. So whether that's stepping back and building a new data product that might answer 80% of questions that run across teams, or an example I had was building a media mix model, even though that might not be something that's top of mind for your marketing team. I will say on the other side of things, as a platform leader, there's a few ways I think about ensuring you're delivering value from your data platform. One of them being usage and, you know, satisfaction or nps. So is your platform actually being used? Are those users satisfied and surveying them and understanding where you stand is a as a platform, are you a platform purely for analysts and data scientists, or are a wider group of people adopting and using your tools? That tends to be the most common and logical way people look at platforms. But then I found that we had to look at impact and maturity as two almost like short term versus long term views on what the platform should build. The impact being, can you attribute dollar values to what your teams are working on and whether that's directly driving through machine learning or whether it's indirect that your teams are actually enabling another team to drive value. If you solely focus on that impact metric, you may find yourself focusing too short term. And so what I've put in place in the past is maturity scoring, where you can think about your platform and various components of your platform as are they just keeping up or are they building ahead of the current needs and then thinking about individual attributes like extensibility of the platform, scalability, reliability, resilience, self service, and actually giving the teams license to work on these things and time to work on these things that that actually develop the platform ahead of current needs.

So you mentioned here at the end maturity score, right. So data maturity is something that we think about often at datacamp because it's a really multi pronged concept for us. I wanted to understand here from you. Do you think that the type of projects data leaders should focus on, do they have a inverse function with the data maturity of an organization? So for example, if you're a highly data mature organization, you can afford more time to be more forward leaning and think about the future. But if you're less data mature, you need to focus on delivering value with data as soon as possible to get that buy in to be able to provide more value down the line. Or is that the opposite? For example, I'd love to see your perspective here.

I do think there's actually two perspectives on this to what you say. One is, and I think about the nine years I spent at the times we really needed to get the basics in play so that we could actually free up the data team to do the more expansive and interesting data roadmap work. So getting in place core self service access for just basic questions so that the data team wasn't reactively focused enabled us to think more proactively about the business problems and you know, tackle them in, in ways that required us to actually build out our data platform to a place where it was possible to answer these sort of bigger questions on the business. But I think on the flip side, when we started to do machine learning, there was probably like a wide space for moonshot type projects because the team was new and they didn't really have any obligations at that time. So you had a, you know, a team of five or six data scientists who were thinking very expansively and consulting across the business and finding places they might add value, but potentially with a very low success rate. But later on in the maturity and as the team grew, once they start to land those projects, those projects hit production. Suddenly, you know, we had four machine learning teams have each had three or so data scientists and three or four machine learning engineers and those were some of the most valuable projects to work on. But you actually limited the amount of exploration that you could then do on new projects.

Yeah, that's really great perspective, kind of showcasing double sides here. Now you also mentioned the second component here outside of proximity to the business, which is how do you transition a team from being a cost center. Right. To a revenue driver? So I'd love in your experience if you can outline, maybe we've touched upon this so far in a few different ways. What do you think are what's a step by step framework for taking a data team to a revenue driving organization? Walk us through patterns that you see of a successful transition here.

Yeah, so I do think and this probably applies more to the sort of mid to large organizations I've been in where you start with a lot of legacy and separated platform. And so I think the initial phase that is critical for you to undertake is that transformation from your legacy where you're very internally focused as a data group and you're actually building out a platform that can make data easily joinable, that can build a fast reliable SQL environment for analysts to be able to work on that has a level of self service that means that the, the level of work analysts and data scientists can do is not simply reactive answering the questions. And so I think that sort of platform phase is most critical and probably the biggest unlock for most organizations in shifting away from legacy and onto say a cloud data environment with the various tools in place for experimentation. Business intelligence I think then as I pointed to before, like investing in various applied machine learning initiatives that aren't individual data scientists but are actually high value problems for the business. Is it how you drive subscriptions for the business or is it a recommendation system that you're putting your big product bets on and you're actually allowing that team space to go and solve that problem over a long period of time and build out the underlying technology to support that at at scale and then I think starting to demonstrate the value from those initiatives. So how are you reporting out the ROI of those initiatives so that you know where to invest and where to actually pull back investment? And there I think it's critical where you have this kind of conceptualization of ROI across the data team that everyone is aligned around.

And you know you mentioned here in one of the sections around the importance of business intelligence and kind of self service analytics, you also mentioned that throughout numerous responses here. I think this marks a great segue to discuss the importance of breaking down data silos and the role of data leaders when it comes to enabling people to fish for answers for themselves. I think one of the probably biggest force multipliers for data teams to provide ROI is, is by providing an insight layer for the rest of the organization to make decisions for themselves and to stop being a bottleneck when it comes to data driven decision making. So I'd love to understand, walk us through. As a data leader, what do you think should be the data team's role in democratizing data insights within the organization and what is the ROI of these initiatives when they are done right?

Yeah, and I would just say to answer the first part of your question, I think it's critical that the data team own this and not have it foisted upon them by marketing teams or product teams. You want to embrace self service? I do think it's an interesting conundrum for data teams because you obviously have hired experts to do data analysis and not necessarily as a support function for other groups. But stakeholders tend to expect straightforward questions to be readily answered and analytical work, for better or worse still carries this expectation of do it yourself in a way that maybe design and engineering don't. But I find like very often if you don't do this, analysts are stuck responding to rudimentary questions that could really better be addressed with just access to high quality, well governed data. And then you can actually free up the analysts. I'd say the first problem for data teams to solve is how to invest in self serve in a way that probably does two things. One is give broader access to trustworthy data to make decisions faster. I've talked to a lot of companies that actually measure that by speed to insight, which of course in its end state could then be tied to faster shipping and product value, shipping features I mean, and then secondly I think a shift in the analyst workload or the data scientist workload away from more service oriented work and towards your kind of data and insights roadmap or project based work. So measuring the impact of the data team and I think it's critical that you see that as the value derived from self service, not just the broadening of access. And you may not necessarily have every decision then be higher quality because you're broadening it into a less trained group of users, but you can at least ensure that that's on a basis of trustworthy data.

Yeah, that's really great. I want to talk about the data quality here, but what's really interesting for me when it comes to self service analytics a lot of the times is that the data team puts in a lot of considerable effort in creating the insight layer, but adoption is not necessarily in their hands. How do you measure ROI of these types of project when adoption is not necessarily in your hands?

So I think, and even reflecting back to my last answer, that time spent on implementation and access is usually well spent, but the payoff in terms of self service needs to be greater than the effort you're spending. Maintain it. You can think of if you implement amplitude or Google Analytics or something, you're actually going to Shift from those analysts potentially doing analytical work to spending time on implementation to enable many more users. And so you can think of what you're enabling there and how many users and the access they have versus the time your team's spending on implementation. What I've found is that the further you get into driving adoption and building this sort of self service access, that equation can be less clear. Right. If you're building out custom experimentation framework for every team, you may start to hit this sort of point of diminishing returns where you're actually better for the complexity of the work just to have it owned by the the analytics team as opposed to enable very complex analysis in the groups. You mentioned how you drive adoption. I do think this is a cultural change. You need to have a very good partnership with your product or marketing partner who equally cares about driving adoption of these tools. You need to see it as a cultural change where you potentially hiring in different skills. I've seen analysts push into product roles as a way to instigate some of that change. And then I do think the sort of embedded analysts can help drive the engagement with those tools and the adoption of those tools as opposed to having a centralized support function for them.

That's really great. And you mentioned here the importance of cultural change. Walk us through your experience, maybe engaging with these cultural change projects. What have you seen are best practices that work for daily leaders when trying to engage the organization to become more data driven and adopt these types of tools? What are important change management tactics that you've seen work in the past?

Yeah, it's a really good question. I think what we did at the times that I thought was quite new for data teams was actually do a lot of research on our users. So we actually had some user research conducted internally to see how they were using data products. We actually had a data product that was used by journalists. So up to 2,000 journalists in the newsroom that were engaging with data every day, that was a data product we owned. And so understanding not just the most vocal and power users of your platform, but actually doing pretty broad research and understanding maybe why some people weren't engaging with that tool so that you can drive that adoption. I think as well tied to that, you know, you have to be credibly building and deploying a data product for them. It's pretty easy to get the wrong signal about data tooling by following an executive mandate or making assumptions around the tooling people need based on how we as analysts or data scientists engage with tooling. And so I do Think that kind of user research phase is critical. Secondly, I've just found, you know, through experience that trust is this somewhat ambiguous concept but imperative for ensuring ongoing usage. And overall, I've found, and found the hard way, that once you lose that trust, it's very difficult to regain and build back up and have a data product that people care about and people want to engage in if they've lost trust. And so I've been thinking a lot about what it means to build trustworthy products and that link between trust and reliability of the data product. Let's take this example of like a product that was used by thousands of journalists in the newsroom, that was used to provide traffic and audience stats on each individual article at the click of a button. We would get notifications from them in Slack as soon as they saw something that didn't look right and they would either question the data or they'd question their own ability and understanding of what we were trying to present. And so we were very reactive to that. But critically, we had a team and a person that was accountable and responding and engaging on their level to understand how they needed to use that product. So I think the first thing to enable trust and reliability is having that accountability and investment in support. I think the second thing is establishing your service levels for that tool or product and so shifting from being reactive to actually more proactively identifying the issues in your data. The a huge thing where something like observability comes into play, but also even more so than having a perfect data product is setting the expectations for uptime of that product and even how rapidly you'll respond to issues. And then thirdly, I just say is active communication. You know, you're putting out a message to your users to say, hold off, this data is not right. We're working on it is so much more valuable than hearing from them or taking the risk to hear from them that they've found a problem in your data products.

I love that example that you mentioned here. And I think it segues quite nicely to talk about the importance of data quality when building trust as a data leader. Given your work at Monte Carlo Data, given your previous experience at the New York Times, I'd be remiss not to chat about that. You've advised you currently in your current role, you advise quite a lot of organizations, data leaders, on how to embed data quality, thinking, data observability into their data strategy. So that trust is part of that strategy. What do you think are main components of a successful data quality strategy here.

Yeah, it's a great question. And I think as we've talked today, data quality is becoming this critical necessity as you're driving more data products that enable decision support, custom facing algos or data products, and then the underlying platform that enable you to scale data product development. We typically think of one of the issues of data quality being downtime, right? Which is erroneous or missing or incomplete delayed data that often plagues these initiatives and can undermine these initiatives. And the consequence of downtime, if you think about it, can range from this almost trivial where you're, you're actually having engineers just respond or analysts respond and it's lost hours to address the issue to actually more existential where you're losing trust or if it's a customer facing data product, you might be directly losing revenue or losing customers. And then at the far end of the scale, you could actually be putting in danger the reputation of the business if it's publicly Wall street issues with financial reporting. So, you know, we've found through some Monte Carlo studies that data teams have around 70 high severity incidents each year per thousand tables and about 30 to 50% of data engineering time is actually spent on fire drills. And so this is a huge problem as you think about supporting initiatives like data democratization across a business. And so at the times I saw it as something we needed to critically pair with a democratization initiative. We needed to actually build in that trust from the outset so that that initiative could succeed. And so data observability solutions like Monte Carlo provide the tools then to detect the issues, resolve them and prevent them, to actually reduce and prevent this data downtime. So I just say across that, for me, it's really hard to imagine investing in an expensive cloud data warehouse or new data platform or data democratization initiative or data mesh initiative, all these initiatives without having that sort of observability in place to ensure data is trustworthy and reliable.

Couldn't agree more. I'm actually very bullish on data observability as a category as well. This was something that's featured in our Data Trends and Predictions podcasts and webinars and the white paper. Now you mentioned here how it's really difficult to not engage in a data mesh investment or a data platform investment without an investment in data observability. There's quite a few innovations that I've seen you speak about while preparing for this podcast. Whether that's data mesh, whether that's data contracts. I'd love for you to walk us through here. What you see as the lay of the land when it comes to the latest innovations, when it comes to tools and frameworks that enable ROI for data leaders, when it comes to data quality and data democratization.

Yeah, and I think, I mean to that question, it is like an interesting time for innovation in this space. I think for anyone that's been around this space for a while, data quality and reliability are forever problems for data teams. But historically I think we lacked a lot of tooling and solutions to address the problems. And so I think you can approach the problems in this space through the lens of how I said we think about observability, which is you have detection solutions, you have resolution, you have prevention. And so just to start with Monte Carlo on the detection side, you have automated machine learning driven monitors, you have ways to target your alerting to different, different teams to make sure you're managing that signal to noise ratio in terms of alerts. Then on resolution you have tools like, you know, I'd say field level lineage is one of the biggest innovations I've seen in the past few years where you can actually look upstream as an analyst. I was never able to do this when I was an analyst or you know, working on projects, but look upstream and see the initial cause of the data incident that you're investigating and be able to resolve it and talk to the right partner upstream. And then also for those data producers to be able to look downstream and see the blast radius of an incident on their side, I think that's just a phenomenal innovation in this space. And then also bringing in obviously logs and other tools to troubleshoot from things like DBT and airflow so you can quickly resolve these issues. You know, and I just think like I've been in teams where we've spent weeks looking to resolve data incidents and now you're starting to have the tools to be able to resolve them rapidly within hours instead. And then finally on the prevention side, you know, some innovations on our side are things like circuit breakers where you can actually arrest the the process and prevent bad data from making its way downstream. And also then insights into degrading queries and things that you can get ahead of and actually improve the health of your environment. You mentioned data contracts and I think this has been a huge topic lately, but it's and one clearly aimed at the prevention side. Right. And a lot of the frustration I think data teams have with the fact that a lot of these incidents come from software engineering teams or data producing teams. And I've had the experience of commerce systems that feed into subscription transformation that feeds into financial reporting. And any kind of schema change upstream can result in many, many reports being affected that you're actually having to chase the underlying problem from. So I think data contracts are an interesting area for teams to look at in terms of can they actually put more prevention in by having upstream data producers manage that not just in form of any changes, but actually work with data teams on changes to the contract that are then going to affect the schema that's being passed through? I don't think they're a panacea. I think in ones like I mentioned, the commerce team, the, that maybe has a low rate of change and one team that you can work with, they're a very good solution. I think in other cases, my experience at the times, it was hard to tell the newsroom that they shouldn't publish an exclusive story because it might change the underlying data schema. So there's some places where I think data teams have to react and adapt and build a bit of separation between the underlying data or the raw data and their actual transformations that they're doing.

That's really great. And you know what I love about the holistic answer that you just gave here is that it really puts a light with the ability of tools like Monte Carlo and the data observability category in general. It really puts a light on the importance of collaboration between different parts of data teams. Do you think with that kind of, with the advent of these types of tools that data teams will have to reorganize themselves around these types of collaboration patterns to be able to respond to incidents better and deliver value faster for their organization?

Yeah, I think that's a really interesting question. As we look at the teams that have taken on something like Monte Carlo, have taken on observability. You're often landing in this centralized data engineering or data platform team and you could be landing with a governance team or you could be landing with a set of engineers who have identified this as a problem. But then you're actually expanding it out potentially to analysts and data scientists or even these data producing teams in more of a mesh type structure. So you actually, I think one model is to actually push it out and have the data owners, you know, the people that are owning the data products that are owning the source and the consumption of data, own the reliability of that data and participate in this process. I actually think the other model that I'm seeing a little bit of, that's more of a new structure, is modeling after how we see a lot of DevOps teams created where they're actually not, you know, attached to any sort of individual product ownership, but they're able to roam across various teams. And I think this is another model where you could have data reliability engineers that are acting in a similar way to DevOps teams and are the experts on working across the stack, working across data products to identify problems and to put in place better reliability, both proactively, but also also be the masters of incident response and resolution.

That's really great. I love that. And Shane, as we are closing down our episode as well, I would be remiss not to talk to you about what do you think will happen in 2023. So given that you've been in the space for quite a while now, what are your expectations for the data and data quality industry this year? What do you think are things that we're going to see?

Yeah, it's a really good question. I mean, I think, you know, there was a lot of talk last year around metrics layer and I'm interested to see how, how practical that gets this year. I think one of the when the rubber hits the road, it's actually where the downstream solutions are ready to adopt these sorts of metrics layers or semantic layers and, and adjust the way they ingest data. Observability is this thing that's going to need to adapt and exist across however the stack shifts in terms of that transformation logic. I think the other area that's really interesting is just how business intelligence is evolving. We mentioned this earlier in our discussion, but we've been enabling these teams to use traditional business intelligence tools. I think analysts and data science scientists are navigating now more towards notebooks, things like Jupyter and Hex and other solutions that are more collaborative and allow for more coding in them. And I actually think there's more likelihood that we'll actually see tools that product managers and other end consumers use just plugged directly into the warehouse so that they can use Google Sheets rather than going into a BI tool. So I think that's a really interesting space and obviously ensuring data quality and reliability across that journey into these end tools is something that Monte Carlo has a mission to do. So I think there's, I'm really interested to see how that part of the stack evolves. I think the third one I'd just say is the machine learning stack, which has largely existed as is this kind of separate, less defined stack and much more homegrown for many teams. But there's clearly some interconnection even at the times. We actually built a feature store that was used by data scientists and data analysts. And so I think there's a lot more commonality across the stack and room to integrate parts of the analytical stack with the data science stack.

That's really great and that's definitely a lot of exciting trends here. Finally, Shane, as we close our episode, is there any final notes you'd like to share with the audience?

I'd just like to say I really enjoyed this discussion with you today and if anyone wants to reach out to me, I'd love to talk about data strategy, data platforms, observability, any of the topics we've chatted about today. So you can reach me on Twitter I'm ShaneM5 or on LinkedIn. I think I'm ShaneMurray5 on LinkedIn, so feel free to reach out there.

We'll definitely include that in the show notes. I also really enjoyed our discussion, Shane. Extremely insightful. Thank you so much for coming on Data Framed.

Thank you.

You've been listening to Data Framed, a podcast by Data Camp. Keep connected with us by subscribing to the show in your favorite podcast player. Please give us a rating, leave a comment, and share episodes you love that helps us keep delivering insights into all things data. Thanks for listening. Until next.