#74 Harnessing the Power of Collaboration with Engineering Manager at Lucid Software, Brian Campbell

Hi, I'm Maggie, VP of Curriculum at datacamp, and I know firsthand how overwhelming it can be to keep up with the rapid changes in data and AI. But staying ahead doesn't have to be hard. That's why our curriculum team has curated the best learning content to help you and your team stay ahead, covering everything from data and AI literacy for busy executives to cutting edge generative AI for developers, and everything in between. Whether you're upskilling yourself or your team, we make learning practical, engaging and flexible. It just works. Power your data and AI transformation journey on datacamp.com.

Hello, this is Adele Neme from datacamp and welcome to Data Framed, a podcast covering all things data and its impact on organizations across the world. A key aspect of developing a successful data science team is strong data science project management. Since data science teams are consistently pushing out solutions that provide value for the entire organization. Collaboration is such an important skill, skill and competency to foster. This is why I'm so excited to have Brian Campbell on today's podcast. Brian Campbell is currently an engineering Manager at Lucid Software where he leads the company's data science and engineering efforts. From helping a three person marketing department configure Google Analytics to being the tech lead on custom streaming analytics solutions, he's built up Lucid's data ecosystem throughout many stages of growth. All of that was not possible without strong collaboration. Throughout the episode, Ryan discusses his background, how data leaders can become better collaborators, data science project management best practices, the type of collaborators data teams should seek out, the latest innovations in the data engineering tooling space and more. Brian, it's great to have you on the show. I'm excited to speak with you on successful data science project management, how data teams can ensure they're maximizing value with their work. How but before we begin, can you give a brief background about how you got into the data space?

Sure. So I started as a software engineering intern at Lucid software about seven years ago. It was very small at the time, 50 people and the team I was on as a software engineer was interested in a broad set of things, but that included helping with the company's analytics. I'd be working with our three person marketing team to set up Google Analytics and Facebook and Twitter, those basic out of the box analytics apps. And I found that I really enjoyed doing that work because I enjoyed talking to my customers. People that were affected by my work were the people that I sat next to every day and seeing their lives get better, their jobs get easier was very fulfilling. And then as the company grew from 50 to now 800 employees. We had more and more advanced data needs. We needed data warehouses, we needed a clickstream analytics system. And since I knew the people, I knew the problem space, I got to take over and lead and now manage that. Of course, then data scientists came into play too, and I got to work with them on their projects.

That's very exciting. You're someone who's led the successful deployment of quite a few data projects throughout your career. You know, while no data science project is the same, I'd love to really discuss the anatomy of a data science project. Do you find that there are common characteristics of a successful data science project within an organization?

So in my mind, success comes a lot from alignment. The more that the stakeholders, the people that want the problem solved, the science team and the team that's bringing the project into the world, whoever's going to be implementing it so that the stakeholders can use it, are going to need to understand everything, be on the same page about requirements, be on the same page about timelines so that they can make that successful. And I found, doing this, that no two data science projects are the same. And the quicker I realize that things are going to be different, the quicker we're able to align. I've come into projects before where we come in and say, hey, we did a project just like this a month ago, let's just do it again. We don't really talk, we don't really do the coordination we did on the first project and then we find out something's different. The new model has an output that's too big for the message bus. The new client has a timeout that's too short for the existing system. And then now we're delayed by weeks trying to figure out what was different when if we had talked about what was different out of the gate, we'd be in a much better place.

I'd love to expand on that, but just before being laser focused on value is obviously a massively important component of a data project. What do you think are the pitfalls data teams face when they're prioritizing projects that lead them to lose sight of the high value projects?

So one thing I've seen pretty often is teams starting from their data, they look at the data that they have and say, hey, I could probably do this cool thing with it. There's this interesting dimension that I could predict. I can do this classification. And then they do it and find out that nobody actually needed that done. It looks cool, it feels good, but it ends up not being that useful.

What do you think are some of the best practices data leaders can adopt when scoping data projects within their organization? And what are some of the key considerations they should make?

So I always encourage teams to start from problems, go out to product managers, go out to your executives or directors, see what problems they're having, find what problems a data scientist can solve, and then start getting your data.

So a lot of the time, the data solution that a data team is developing is a solution to a business problem, as you just said, by working with the problem experts, for example, do you find that the data solution can often evolve throughout the project life cycle? As in, you start out with a very specific solution in mind, like a machine learning model, and then end up building something completely different. But that solves the business problem at hand?

Absolutely. A lot of the problems that my team has set out to solve with an advanced technique have just ended up as a visualization or a dashboard, some way to make the data more accessible to whoever the stakeholders are. They don't necessarily need these big fancy prediction models out of the gate. They just needed to see their data in a new way, in a more accessible way. And by doing that sort of 20% of the work, you get 80% of the value. But then you can start building on it. A great example we have at Lucid is with our anomaly detection work. Anomaly detection libraries these days have huge numbers of bells and whistles that you can use to tune your anomalies. But we found that it was better to instead of trying to get the perfect curve and finding the right anomalies, it was easier to get something out with the default models. Start seeing how people would use the anomaly detection, find which ones they cared about and which ones they didn't care about. And from there we started doing the math to add custom regressors, get the right seasonality. It turned out if we had done all that work for something people didn't care about, we were getting too many payments. It was great, but nobody needed to respond to that. We didn't need to monitor that. We just needed to know when we weren't getting enough payment, when payments sunk.

Okay, that's awesome. And what do you think are some of the lessons maybe you can share or best practices that data leaders can adopt to ensure that they're always finding the best solution to a problem?

Yeah. So always start small. Always start by getting something in front of a person as quickly as possible. If there's a baseline model you can use, if there's a simple visualization you can build to try to take care of this problem. You start there, see if it helps, and then go to the next bigger thing, the next more sophisticated thing. Don't try to start from your sophisticated answer, try to build towards it and validate it along the way.

Keeping on the same theme. Oftentimes, with any data science, specifically machine learning project, common evaluation metrics used are metrics like accuracy or recall. However, in the real world, algorithms are evaluated not only on accuracy, but on robustness and the value they generate. So what do you think are important metrics to track throughout a data project's life cycle? To ensure teams are maximizing value?

The metrics of the business are the ones that should always be at the forefront of the team. If you're working with a product team, you need to care about product metrics the most. You got to care if users are coming back to the products people are purchasing, if they're having an enjoyable experience in the product. And then if you're working on a business issue like marketing cares about CPC sales, is going to care about closed deals. While data tends to be one or two steps away from that, that should be your North Star metric. And then you can build down to see how what you're changing, how your accuracy affects those metrics. And then while you're doing that, you can find interesting secondary metrics. We set up a system recently where we had a document clustering problem. We'd get some data from users and we'd cluster it for them. And we built the best model we could. We had a loss function and we optimized for it and we found out that it was, it took 30 seconds, we thought that was fin, we put it in front of customers. They were not willing to wait 30 seconds for their clusters to come back. They would bounce from the site before we were done. So we had to come back through, change the algorithm. And while we were not as good at having the best clusters, now I've seen a lot more users happy because they were taking just a couple seconds to get the experience they wanted. And another interesting example I have here is from the E commerce space. There's a team working on making product recommendations. So you're in a store, you get recommendations in Amazon all the time. So they were always trying to increase the relevance of the recommendations. They wanted people to purchase whatever they were showing them, but they found that there were some trade offs there. And if they increased the number of people that would, they'd recommend pens to people. They increased the number of people buying pens. By 10x, that's great. Mandations are more relevant. But if that model suddenly was now selling like expensive electronics less, the company was losing money. So they had to pay attention to what they were recommending and making sure the best products for the company were also being recommended.

And how do you go balancing out any potential friction, maybe between the metrics the business cares about and the metrics the data team may care about?

The best method, I have found is to encode the business what the business needs into some sort of test so that when you are out there doing cool stuff to optimize your model, you or your scientists can focus on optimizing the model. Then they can come back and check it against the test that you've written. Hey, for this input, it's very important that we get this set of outputs. Like in laptop example, we could be optimizing our model and then check it against common use cases to make sure that things that are important to recommend are still being recommended in an automated fashion. So it comes from there. You have to just have the right partnership to understand the business, what the business needs. And a lot of that is learned by hard experience. You won't necessarily think about these side effects in the first iteration, but as long as you're able to iterate quickly, you can fix it and move on in a second, third, fourth iteration.

So given your engineering background, I actually love your perspective on quality and availability. So data availability and data quality are also super important factors when deciding on which projects are high impact enough to go on a data team's roadmap. What are the baseline characteristics of data for a given project that give you confidence that this is something that can go on your roadmap?

So when it comes to putting something on the roadmap, what's really important to me is, well, when I lead my team, my engineering team, we get a request for data, we don't say, hey, is this available? We say, okay, here's when we get it available by almost all data can be gathered that you need. So having the engineering team think about how they're going to gather it is what's important. So that's why you've got to get them in alignment quickly, because they might come back and say, we can get this data, but it'll take a year to build a new pipeline, to build a new control. So maybe this data isn't going to be available in the time we want. But then that's a choice you have to make the team itself, if you're a data Engineering leader, try to focus on making anything you can available so that when a data scientist comes in with, hey, we've got this high value project and we need something new, you can go make it happen.

And what do you think are some of the common blockers data engineering teams go through in trying to procure data silos?

Silos are a big one. Hey, we've got data in some third party. That third party belongs to the marketing team. You've got to go figure out who in the marketing team is going to get you the access you need and then figure out how we're going to interact with that third party without disrupting the marketing team. We use Marketo at Lucid. Marketo of course has a certain number of API calls. We purchased an API call limit from them and if we're pulling data too fast, suddenly the marketing team can't do what they do. They don't have enough API calls. So we have to find this right, trade off between getting the data we need while not affecting the actual business users, then just novel formats. I think it's a common problem everybody has is, hey, you get data in a format you've never seen before. It's going to take a while to.

Figure it out how agile iterative data teams should be. What are some of the tactics you found effective when working in an agile iterative manner? I'd love it if you can anchor some of those tactics in some real world examples.

So the most important thing I've learned from Agile and getting that to data science is to start with an MVP and to build build on the MVP quickly. I find a lot of data science projects fall into the waterfall trap of we're going to create the perfect project, the perfect thing, and get that out in front of customers when really you can start with something simple and naive and start learning about how your customers are going to interact with it. So if we go back to the example from before about document clustering, to give some more context there this was, our customers can come into our app and do a brainstorming session. They put stickies onto digital whiteboard, put ideas on it, and then we cluster those ideas into kind of topic groups for them so that whoever's moderating the brainstorm doesn't have to lay it out themselves because that can take 10, 20 minutes. And so the first thing we did was we created a model that just returned random clusters and that allowed the product development team that was building the user experience around this feature to start getting a sense of what the user experience should Feel like they say, hey, we're going to send stickies. This way the API is going to return the clusters in this format and they could start building. So we didn't end up in a place where when we were finally done, they could start building, making it take that much longer to get in front of customers. And then once we had the algorithm we wanted in place, we set it up behind the API that the product development team had already built and the product manager started playing with it. But they were getting weird results. As I mentioned, it was taking a long time and they were not getting the results they was expected. But we didn't have enough data points from just the one product manager to see what was going wrong. So that product manager suggested an internal alpha. Okay, let's take this clustering feature, let's put it in front of other people in the company, see what's going on, see if it's just me or if there's something wrong. And turned out there were some things that were really long with the algorithm. There were lots of weird edge cases with the language processing we hadn't considered. People were submitting stickies that only had stop words in them, or there would be stickies with essays and a real life brainstorming session that we got to participate in. And we saw them. They brought us a document that had hundreds and hundreds of sticky notes. While we were expecting in all of our test cases on a data science tie, no more than 100. And so that would we'd like. We were running out of memory, we were breaking down really quickly. And so being able to go to a customer that's around us, see their problem, fix it really fast, put it in front of them again, we're able to iterate really quickly and then get to something better, something better for our customers faster than if we had tried to fix everything up front. And of course, there were things we never thought about that came out of this internal alpha. And then actual customers would give us feedback too, and we'd fix it and we'd continue to iterate that way.

That's great. And I'd love to dive deeper into this and discuss the monitoring aspect of a project. So, you know, a lot of the times data teams work on a model, deploy it, and then problems arise in production. So how important do you view monitoring in mlops? And who do you think should own it?

All right, so monitoring is incredibly important. When we create a model and release it into the world, our intent is to change the world in some way. And so our model by definition becomes inaccurate very quickly. But we also have the world around the model changing as well. We frequently see marketing changing who it's targeting. And so our customer mix is going to change. We're going to see third party APIs change the way they work. We have one model that's based on Zendesk. Zendesk changed their API and we didn't have the right monitoring in place to detect that we weren't getting the same data format as we were before. And thus we weren't getting the results we were getting before. Until somebody brought it up much too late that they had stopped seeing what they had expected. That's really a situation you don't want to be in. So getting that monitoring in place around your accuracy, around your business metrics is very important in terms of who should own it. Difficult question because you need someone, you need a point in your organization that is an expert in kind of your infrastructure for monitoring infrastructure and an expert in the data science side, expert in the product side. We've built up a data infrastructure team that combines the need for data engineering and these data pipelines, as well as support for data science. And that's been the team that's owned monitoring so far and that's worked for us. It's an engineering team that kind of cares about everything, so they can put that monitoring in place. But it's a difficult skill set to find and difficult skill set to train, so we're all working towards it together.

You know, given this is a relatively nascent space, what do you think are some of the tools that kind of help teams monitor models in production?

So we inherited from our SRE DevOps team Datadog, which is mostly a tool for, you can throw in numbers at it and then monitor how those numbers go up and down. And just the ease of doing that, it seems like a simple thing to do, but doing that at scale is very difficult. So having a tool that handles all that for us is nice and then it notifies us via Slack. Or we can set up for very important things. We can set up an alert through something like PagerDuty to tell us that something is broken, it needs to get changed. But honestly, we've had to build a lot of stuff ourselves as well. It's not as mature a space as DevOps, so there's going to be a lot of kind of hacking things together yourself as we write, we write airflow dags that monitor systems and then do Slack notifications or kickoff retrainings.

Are there any particular tools that you're.

Excited about One thing I've been impressed by are tools like they're on the kind of the input side rather than the output side. There's tools like Great Expectations or Monte Carlo I know was on the show a couple weeks ago that let you keep a really good eye on the data that's coming in and making sure it doesn't change. And that one can tell you that you need to retrain if it changes too much or at least can tell you something has changed and you need to go figure it out. I really am looking for tools that keep an eye on output as well, making sure that what your model's outputting is still what you expect from it.

I'd love to segue here more to discuss how the data team fits within the organization and how important for data leaders is it to build relationships with the rest of the organization so that they're consistently iterating on high value projects? Obviously, a lot of the high value problems waiting to be solved by the data team sit within functional business units. So how can the data team establish trust and friendship with these business units so that these business units become strong customers of the data team's solutions?

I'll be honest up front, I don't have a good strong process for this, and it's maybe because I haven't worked out enough, maybe because it's a much more human problem than what I'm used to dealing with in engineering and data sciences. But there are some things I found that work. So first thing I found is that people like talking about the problems they're facing as long as they're interesting. And so taking that time to take organizational leaders to lunch or to a coffee and just letting them open up about what they're dealing with will give you a lot of insight into what's going on and build a lot of trust by showing that you care about what's going on through the company. Once you have that catalog of problems in your company, then you can come back with your data science team or analyst team and say, hey, which of these problems can we would be good for us to solve which of these are inherently data problems? From there you can come back and say, hey, I thought about your problem. I think we can put a solution together. Who's the right person in your organization to own creating a solution with us? And now you have a relationship that's more formalized and you can start going towards building your solutions.

And how do you go about prioritization based on the problems? Do you integrate the functional leaders in the prioritization process or is this owned by the analytics team entirely?

Yeah, the data team tends to own the prioritization side because we might have to do some extra footwork to figure out, well, how valuable would it actually be to solve this problem? But that's only one axis of the prioritization. The other one being, like, how feasible is this? Can we do this? Would it be worth doing? That's where you really need the data team to be involved.

You know, the power of collaboration here also extends to implementing a data solution for any specific business problem. Can you comment on the steps it takes to take a data product from scoping to actually being embedded within a business process and the folks data leaders are expected to work with to make this a reality?

So I'm assuming you've already kind of done the steps I've talked about, you've found your product, your problem, and you found the expert that's going to be the person that you're validating your solutions with. Now you need to figure out your data. What data do I need to create solutions to this problem? I need to start working quickly with who's going to get you that data, Whether it's an engineering team, whether your team's going to have to work with a security team to get permissions to access something, whether you have to work with some other team to get into some data silo. And you got to start making those friends quickly and start ingesting that data quickly, because data is what makes this whole system work. Otherwise you would be having a data problem from there, if you have enough, I like to have enough scientists, but there's usually more work to go around than scientists to have. Well, one person is playing with the data, another person is kind of exploring solutions and finding different things we could do. Once we have all the data. My data engineering team, we try to get at least some of the data quickly, some small subset, so that as data scientists can take a look at and explore the format, what dimensions they have available and how they're going to need to clean it before they have the full data set, which could take days or weeks to get in place. Then once you start kind of forming around certain solutions, you have a good idea of what you're going to do. You need to find the implementation partners because they're the ones. Data science teams really have the full skill set to take something, take their model and get it in front of customer. They need some help from some someone. If you need, you say, hey, we're going to create an API, you're going to need someone from your ops team or your platform teams to create that API. If you want to add a feature to an app, you need a product development team that's going to be the own that UX and these teams all have their own roadmaps, they have their own goals and you have to find a place where you fit within those goals. So making sure you have your solution pretty clearly delineated so they're not going to have to do a lot of fact finding for you and helping them understand that what you're doing is very valuable and thus it is good for them. It is just as good for them to help you as it is for you needing the help. Like, everybody's going to win if we put out something really cool. That's the point where you understand who you're going to be working with. You've started to build solutions. It's mostly now about keeping everyone aligned, keeping everyone in the know about what's happening. Data science projects, the timelines are messy. Like I worked in engineering for years before I got into data science and those timelines are already messy. And data science has an extra level of complexity on top of that that makes it harder to pin down. Hey, we're starting this day and we're going to be done on this day. Like model selection, model building always takes longer than you expect, but you still need to communicate with all these partners about what's going on. So I find it helpful to set up clear milestones and at what milestone we going to need our partners to get involved. So they know, hey, we're at this point, it might be a week, it might be two weeks when we're at this next point, but that's when we're going to need you to start helping out. So they know at least a few weeks ahead that they're going to be involved in this project.

So you mentioned here the timelines of a data science project are especially messy. Do you find that there are some best practices data teams can adopt to gain some automation on common data science tasks to reduce the workflow time?

We're starting to see really cool stuff in cleaning automation. AWS put out their cleaning tools. We've only had a chance to plan with them. We haven't productized them yet, like put it into a production project, but it's a space that's coming and it's going to hopefully speed things up and make things more reliable, which is really what data science needs to become more valuable in organizations.

What do you Think data science needs in order to be more valuable throughout the organization.

In my own organization, I find us dealing a lot with skepticism because we're pushing boundaries about what's possible. People have a hard time believing that we're actually going to do it. Like we set out on this topic. Modeling adventure of hey, we can take the things people wrote and put them into groups based on themes. Is something so new to computers that people that have been working in computers for like even doesn't even be that long. And like that feels like a limitation, feels like something you shouldn't be able to do. And so getting cool things in front of people is how you get them to believe that you can do valuable things. And as you get people to believe in you, they trust you with bigger and bigger projects.

Do you find that high levels of data literacy are important to foster within an organization in order to have a fruitful collaboration with functional leaders? What is the data team's role in educating the remainder of the organization?

To the extent that an organization is data literate, you're going to have these better collaborations. But I found that it's hard to put the responsibility of data literacy on other teams. It's really up to the data team to make data as clear as possible so we can communicate about it as clearly as possible. Yeah. So that comes down to one thing we've seen a lot of success for in our team is in the visualization space. Our sales team, our executive team, doesn't have time to be playing with spreadsheets of data. We need to find what they need to know and turn that into a visualization and tableau or a custom application for them to explore this data in a way that is accessible to them. Yeah. So I would say that the data organization isn't about making the organization more data literate, but making data more accessible to the organization so you can talk about it, so they can make the best decisions they can.

What do you think are some of the other tactics data teams can employ to further enable access? You know, something that comes into mind is data discovery tools, for example, or metadata management tools that let everyone search for the data that they need.

Absolutely. Our analytics team, who I'm not as involved with, but I've been, they're just a rock on team. They, they put together, they call it their metrics catalog. It's a pretty simple webpage, but it lays out, hey, here's all of the metrics that we track as a company. Here's how we calculate it. So if you want to explore that calculation. Here's visualizations, here's SQL queries. If you want to jump into the data warehouse and start exploring that metric, they have thus this piece of metadata that's in a very accessible format, it's not just in a table, but a step beyond that, that gives people the ability to jump in at the level they're comfortable. And I think that's helped to increase visibility of our data and made it more used through our company.

So data teams are often organized differently from one organization to another. You have centralized center of excellence style data teams, embedded data teams where data scientists are embedded in functional groups or something in between, like a hybrid model. What do you think? Organizational model that promotes alignment between the data team and functional experts and business leaders.

Yeah. So my data science and data engineering teams are currently a centralized team because it's very small. We have three engineers, three scientists. We could have a whole other conversation like how do you get anything done with that? So few people. It's been an adventure. But I can say looking at our analytics team who I just talked about, the director, Tim Jenkins has come up with this question, this awesome model for getting his analysts into the organization. It's a hybrid between the centralization and the functional groups where there's a central team that's focused on improving the data quality overall. It's interested in improving the systems that we have in place as well as for handling requests that fall through early in his organization's life. Like the HR team, the people Ops team didn't have a dedicated analyst, but they had an interesting request come through, so it came through to the central team. But he also has teams embedded in marketing and sales and product development that are able to meet those teams needs specifically. But he doesn't go to the level that we see at like Google where you have data people on the individual teams themselves. These analysts are working at more the organizational level. There's a lot of value there because their customers then are the organizational leaders and so they're able to get involved at the division level and the roadmap level more.

This is really interesting. And given your experience working in a lean startup, what are some of the best practices you've adopted to enable the data team to stay hyper focused and laser focused on value while also keeping a sense of work life balance within the team?

Yeah, so the one thing that's been game changing for us is serverless solutions. The one thing that's let us put out models, put out visualizations, maintain pipelines with just a few people has been that we don't need to maintain our servers. We can put something in a lambda. We can put something. We can write a container and put it in AWS Fargate and then not have to worry about the maintenance. I recently was talking to another company where I was hiring a senior data engineer. Talking about kind of what they were looking for the field and they were looking for someone that was able to keep their, their Spark cluster running. And I was like, I just paid databricks to do that. I don't have to worry about bringing on a whole nother head to my engineering team to take care of that when I can outsource it to an organization that's doing an excellent job.

And do you find that the tooling space is making data scientists more effective and productive today? What are the tools that you're excited most about that will make the data science workflow much more efficient?

Thinking through where's my team spent time now that I wish they weren't. There's quite a bit we're seeing developments in automating of model deployments. Right now we have a fairly manual model deployment step where we're running scripts ourselves on like our laptops on host. We SSH too. But we're seeing a lot of as well as having to build something to deploy. Like we have to have a platform team build an API. But we see stuff like SageMaker that's push button. You get a container, that container is running, it has an API. You just call that endpoint with your data and you're getting your results in that. That could make a huge difference for it. For a small team that doesn't have the expertise in setting up servers, setting up APIs and data pipelines are also getting lower and lower maintenance. We've got, I'm at an AWS company, so those are the tools I know the best. But we have AWS glue creating the serverless ETL solution. So there's now another layer of maintenance you don't have to worry about.

It's really exciting to see a lot of new tools like centralized data warehouses, data platforms and all of that fun stuff that really reduce the barrier to working with data and make the data science workflow much more effective.

Yeah. And I know like a lot of these tools are using Spark or the principles of Spark under the hood and it's so nice to not manage that myself. These like Spark is an incredibly difficult tool to keep running on giant clusters. So having somebody else do that makes it so my team can focus on what my company needs rather than Just this maintenance.

So given that successful data science within an organization goes beyond hard or technical data skills, what are some of the most important skills practitioners and leaders should cultivate to become better collaborators?

Yeah. A theme through this has been about communicating with your collaborators, and so making sure that you're communicating in such a way that you are understood. And even with dealing with somebody that may not be as good of a communicator, that you can still work to understand them is very important. And for me, like the key text, there has been crucial conversations. It lays out dealing with difficult conversations, places where the stakes are high. But the principles it has about finding shared meaning, making sure we understand each other, are so broadly applicable, I definitely recommend it. And then on the other side, another aspect here is making sure that you're running your projects smoothly. My data science team doesn't have a trained project manager. I imagine very few of them do. So just understanding the basics of how to take a project, get its requirements, breaking those down into a timeline with milestones so that you can communicate about that, is very important to maintaining a relationship of trust throughout the project.

And how often do you think teams should be communicating with stakeholders Coming kind.

Of being born from an engineering organization? We have two weeks SCRUM sprints, and at the end of our sprint, we have a review and acceptance meeting. I think that's pretty standard in SCRUM to go over what we've done and where we're going next with our stakeholders. And you got to find the right cadence for your organization. But I find through the industry, two weeks is very common.

Finally, Brian, do you have any call to action before we wrap up today's episode?

Yeah. So, like I said, the theme through a lot of this has been about collaboration, building those relationships. And so I would like to challenge all of you to go and find the collaborators that you're missing, who you're going to need help from in validating a solution, who you need help from in releasing a solution, and start building that relationship. Go get a coffee or take them to lunch, see what's going on in their world, and start expressing that you're going to need their help finding that common ground.

Thanks so much, Brian, for sharing your insights and for coming on the show.

Yeah, thank you, Adele. It's been great to talk with you today.

That's it for today's episode of Data Framed. Thanks for being with us. I really enjoyed Brian's insights on the importance of collaboration and data science project management. If you enjoyed this podcast, make sure to leave a review on itunes. Our next episode will be with Andy Cotgrave, technical evangelist at Tableau. I hope it'll be useful for you. And we'll hope to catch you next time on Data Framed.