AgileData #24 - The magic of Agile Data teams vs Agile Software teams - Lynn Winterboer

Welcome to the Agile Data Podcast where we talk about the merging of agile and data ways of working in a simply magical way.

Welcome to the Agile Data Podcast. I'm Shane Gibson.

And I'm Lynn Winterbore.

Hey Lynn, how's it going? Thanks for joining me. You're over there in sunny Colorado and I'm here in almost sunny Wellington. Yeah, and it's been a long time. We have caught up and talked about all things agile. One of the things we do when we have new people on the show is get them to probably give a couple minute background about who they are, what they've done in the world of agile, just so that the audience can get a bit of background about yourself. Tell me all about Lyn and your world of agile.

Okay, thank you.

I'm an agile coach. I have been working almost exclusively with data teams for the past 10 years. And my agile experience goes way back into the 1990s when I was a software developer for a consulting firm that used an agile approach called rad Rapid Application Design. And that was my first job as a software engine engineer. And I thought that's how all software was developed. But then I went into data warehousing in the late 90s and was very stunned to learn that that's not how all software or data projects go. And ever since then I have always been interested in both data and agile. And for a large part of, not a large part, but a portion of my career, I had to kind of switch back and forth, be on an agile web based project that was tons of fun, but I miss the data and then go back into the data side where my heart lies, but miss the agile. About 10 years ago I decided I was only going to do work that intersects agile and data. I have been focused on it ever since.

Cool, Lynne. I think we first met online when I attended a TWI training course remotely from New Zealand. I think I got up at 1am to attend it. It was all around how to use AG with data with a whole focus around testing. And the reason I attended that was it was an area that I knew in the work that I was doing that I was very light on. And a lot of the teams I worked with was light on is how the hell do you apply testing techniques when you're working with data. And what I found over the last couple of years is actually there's lots of agile techniques that are applied when we're doing application development that we struggle to apply in the data space. And for some reason agile and data just seems to Be harder than agile for other stuff. I've never worked out if it's just us or not. I thought what we'd do today is a bit of a chat. With all your experience on why does agile and data teams seem harder than agile in other teams?

I have several opinions on why this is different. I think.

There are a couple reasons. One, I do think that people in the data world and people in the software world kind of went our separate ways in the 90s and 2000s because we had different problems to solve. A lot of the While data people were solving really interesting and unique problems related to large volumes of data and moving data and data cleansing and data quality and data governance software teams were working on how to be more effective and efficient at doing software. They were able to come up with a lot of the.

Practices that are now part of a continuous integration practice. That really makes sense. I remember being in a.

Website project, I think in 2003 where we were an Agile team working in an agile way and the team checked in their work very, very frequently throughout the day into the branch and we did a build at the end of every night and then nobody went home until the build was working great. And that was kind of bleeding edge for that time. But it was not that hard for the teams to do. They had the technology they needed, they had the knowledge to do it and they were successful at it. And yet there's still many, many data teams out there today who aren't able to do that. Now there's a couple things. One of them might one thing might be. One cause might be the.

If we use.

Some of the data management tools that we use are kind of black boxes, they're not really meant to be.

Have little bits of little changes made and then checked into a main branch kind of concept. And so it's kind of hard to figure out how to do that. We also a lot of our architectures are pretty monolithic.

And it served a really good purpose for a long time. Sort of the days of they will build it and they will come data warehouses. We're trying to meet all needs at once with these big complicated architectures. But they were very powerful. They provided some power at that time that was needed and by our business customers and we're often living with some of these architectures were so robust they're still around 20 years later, but they are pretty monolithic. So it's very hard to make a small change to a small piece and not have to regression test the heck out of it.

I think that's Part of it.

Yeah. I think those are two main things. What else comes to mind for you?

I think those are really two good ones. I agree on the tooling.

I think we've seen a change in the market over the last couple of years where the tools that we use to do good and bad things to the data to transform it less black boxy and more templatey. So we're starting to see some capability for those tools to natively integrate with repositories like git. And the idea that as I write a small bit of code, I check it in. When I need a bit of code, I check it out. I think there are still parts of our architecture like our bi reporting and visualization tools that aren't there yet. The clicks and the tableaus of the world still aren't really natively integrated to those types of repositories. There's no concept of me being able to create a graph on a dashboard and check that graph in and then create a second graph on that same dashboard and check that in and treat them as two different objects. So I think, yeah, our tooling is behind the agile techniques we use. Same thing, I think, around the model of architectures in both the technology and platforms we use and the things like data modeling techniques we use. I think we were just served by the big data bollocks that happened, where we were starting to deal with some of the issues around modeling and those types of things to become more agile. And then there was a shift in the market as everybody moved to big data is the next big thing. And so we spent a lot of time dealing with that before we could get back to our craft. I think one of the other things that I am always really cognizant of is when we have a team that's building an application, they're in charge of the data, they own it, they control it, they control the structures of how it's created, they control the screen of how a user manufactures it. If there's a surprise in the data, it's a surprise they've introduced. When we work with data, we have absolutely no control.

We get surprises on that data that's manufactured by somebody else every day. And I think it's that lack of control of what the data looks like and how it behaves at the beginning that causes us a whole lot of headaches. That makes it hard for us to apply some of the apps techniques to the data.

Yeah, that's a really good point. It reminds me too of one of the biggest tactical hurdles we face. Is that our testing environments go back to that testing class where you and I met. Our testing environments aren't owned and managed by the data people. They're owned and managed by the application people.

We need very specific data sets in the testing environment.

To be able to predict what the outcome should be so that we can test. Do we actually get the outcome we need when we have a bunch of junk data in there or we don't control what data is in there? It's really hard. I think in that testing class where we met, one of the ideas presented is that data teams should have their own testing environment and they should be able, like a data team should be able, to control what data is in it to the point that in between each test.

If it's a regression test, the entire schema might be torn down and then rebuilt from scratch in the next regression test with just the bare minimum number of records loaded for that test case. That's an idea that some teams use. The idea to data teams of tearing down a whole schema is just.

Gives people heart attacks because we've never lived in a world where it's okay to tear down a schema without bad things happening. And so the testing environments, we're sharing a single testing environment for two different purposes and we seem to always lose.

And I think some of that is the constraints we had around volumes of data.

Yeah.

In the good old days, we didn't have the cloud technologies we have now the cloud infrastructures, the ability to scale on demand and scale back. So we were always constrained. The idea of running test suite over the last 10 years worth of data in a warehouse was unheard of because it would be the rest of our life. When we waited for the database to respond, I think that's changed slightly now. I think.

The ability to do some of the things we couldn't do before has become available. And I think we're starting to see teams think differently about that. So some of the. The principles or practices that we had that have been around for 10 or 20 years are starting to get challenged now because they were based on constraints that are no longer true. But as all things, when you have a craft, there's often sacred cows and it's done that way and every time you tried it before, you failed, so you're not going to try again. So it's that idea of teams, from a data point of view, being agile and constantly challenging the assumptions and constraints they had before to see whether they're still true.

I think the other thing is.

Apps teams have the ability to have a conversation with a stakeholder slightly easier.

They can mock up the app, they just do the wireframe and they use that as a feedback loop. And then somehow they can craft the bits that support that easier than we can. We can craft a wireframe of what a dashboard might look like or a data service. But the amount of effort we've got from taking the data as we get it to making it fit that wireframe seems to be a lot more effort than in an app team. And I think that complexity in the middle that we have to deal with means that we try and explain that complexity to our stakeholders, to our users. And that makes it difficult because we're talking techno data, speak to them and they don't understand it. Teams trying to figure out how they can have agile conversations with their stakeholders, where they can articulate the complexity in a way that is understood without actually describing the complexity, I think is a bit of a challenge.

I think you're absolutely right. I'm thinking back through, I don't know, probably, I don't know, maybe close to 60 or 70 clients I've worked with in the past 10 years and.

Maybe 10 or 12 years. But.

It is very rare to have a true product owner. If the team is a scrum team or using the product owner construct, it's more often that you have what teams will call a technical product owner. Now, I am not a fan of that. I still push them, I still challenge them. And to say you're still insulating yourself from your customer, you've got to find a way to get out to your customer.

And it's a really hard.

It'S a hard leap. And like you said, it's a habit. It's something that's been there for a long time. It didn't make sense to go to the customer with.

Technical babble, but.

But the teams still need to understand the end use of that data, the end benefit of it.

It's quite empowering, frankly. Some of the teams that I've worked with recently have really embraced behavior driven development and using the Gherkin language to describe the acceptance criteria of a story or a feature and a feature or a story.

It's been really neat to see them talk about the impact.

The resulting impact of the effort. If we do this, then we'll get this result and the result is put in some context that has business context.

That's been really neat, seeing their eyes light up and seeing the product owners say, okay, yeah, I've been feeling kind of bad. I've Been talking in bits and bytes for a while now I get to talk. In terms of business value, we had a big swing away. Let's see, a group that I've worked with recently, they had done a big agile transition and they were using user stories until they, I guess maybe they weren't using them right. They had a lot of like, as a data developer, they weren't really using them right. So then they swung toward just not using user stories and just going back to their old requirements. When I started working with them, I just was asking them lots of questions about that. Some of them were willing to try going back to user stories and we did some user story writing workshops where we really looked at the users. It was really empowering for those teams to realize, yes, here's who benefits from getting this data and to get it down to our customer's customer and tell a story of why this is important and why is this taking up your valuable time as a developer now when you could be working on something else valuable.

So going back to user stories and really working it through to be true end users, not other developers, and then also using behavior driven development and the Gherkin language for expressing acceptance criteria has really helped them reconnect with the business and the goals of why they're doing this versus the technical details of it.

Yeah, look, I'm a massive fan of behavioral driven development and particularly Gherkin. So with some of the teams I've worked with, we start off using the Gherkin framework to describe tests and then we morphing as a way of using Gherkin describe rules. So working with the product owners to use the given and and then framework of what rule we need to apply to do something good or bad to the data. Yeah, like a pseudo language that most of them can understand. I'm interested on that technical owner because it's one that I'd struggled with for a long, long time and I'm still working on. Like you, I originally, you know, identified this idea that you needed a technical owner or an architecture owner that was outside of the product owner. Primarily because when we're doing things well, our product owners were actually business stakeholders.

They knew who to engage in the organization, how to understand what was important and prioritize it for the team, how to articulate what the definition of done done was going to be and the value that would be actioned off the output that the team created. So they behave like true product owners. And typically they had minimal data background or technology background. So when they were given a Choice of, well, do you want us to do continuously deploying CDC changes off the Oracle database or is overnight snapshots okay? They looked at you blankly and a technical person would jump in and become the babelfish, right. They'd become a person to help the product owner understand the consequences of the decision they were about to make. So I was a great fan of that for a while because it was the best I had. Lately I've been doing a lot of research and work around this idea of a product manager.

And so the way I articulate it now is our product owners should be focused on the data product or the information product we're delivering. So you know, it's a piece of data going to an audience in a certain way to achieve some kind of organization outcome. But typically we're also building a data platform, a product.

We want automated testing because it makes us faster and safer. This idea of having a product manager that's focused on a platform, on a way of working on a capability that effectively, if you think about it, something you would like to sell and has a bunch of features, and then the product owners are focused on the data and the content and the visualizations that have value for the organization.

Experimenting with that as a concept, does that work by defining those two different roles slightly differently. And then what happens when we get tension? What happens when the product manager go, actually, no, we can't cheat on this one because the technical debt is going to be too high. Do they have the conversation with the product owner or is there a product owner or product owners haven't answered that one. But for me, that concept of a product manager is coming out of the agile apps world. It's been around for a while and it's not something that they us leverage in the data world a lot.

I agree with you. We don't leverage it that much. I do really like the idea of product management of our data applications as products. And I kind of, as you're talking there, I was sort of imagining that product manager is the one who's maybe on the hook to make the whole platform successful. Right? It's quite a charge. It involves, if you think of a.

Product manager and other types of products in the world, they look, they look internally to capabilities and they look externally to what the market needs. They look at their competitors.

A product manager role, take it outside of data or even an IT role. A product manager is a really intense role. It's a lot of responsibility, but it's a lot of ability to make a difference in the world. And so if we pull that in, I do really like that idea. And I could see product owners.

Being responsible for.

Instantiating certain aspects of the overall product kind of at a more detailed level. But I think it would be very, very healthy for data teams to have a product management view and to have somebody taking that product management view. And it is a blend of what is internal capability, what is going to create sustainability for a safe future.

What does the market need, what are the competitors doing?

And when we talk about competitors of our data products.

A lot of people in IT shops, right? If you work for a large corporation and you're working on the data assets of that corporation, you might think, well, there are no competitors. What I laugh about is, okay, there are a couple different kinds of competitors. One, what do your customers in your organization do if they don't get what they need from you? You have lots of competitors in your organization. And then two, what are your organization's competitors doing with data that makes them better at serving your customers needs than you are? So there are a couple different levels of competition that we really want to look at and evaluate.

And yeah, actually as you're saying that, I was thinking if we think about that product manager in terms of the platform and the capability, there's often a lot of tension. And teams I work with build versus buy. I've seen a big shift in the last couple of years where the majority of the teams I've been working with are building their own platforms, using open source components or cloud services and kind of putting it together, architecting solution rather than buying one. And often.

That'S their natural behavior. So I often see them decide to build something that naturally I would have suggested they bought right as a component. And you're like, well, actually that one's really not a competitive advantage within your team. Right? It's right. There's stuff out there that's available that you could integrate easily. It looks like it's cost effective. So I probably question your build everything. I wonder that's something that the product manager would be looking at is competitive products within the stack that the team are using. So just challenging. Okay, well what are the options? What are the competitive services we could deploy within that product and when should we switch one in or out? When is it worth changing it because it's done its dashboard. So I think that would be a key part of that role. I think the other one is product roadmap. So you know, when organizations start this way of working, when they start their transition, and I love the fact that you use the word agile transition and not agile transformation, thank you very much.

We're not butterflies, right? We don't go into our cocoon and.

Those big shiny suit consulting companies and then all of a sudden we're agile.

Working. Right.

It's a generational thing. It takes time. But we need to set a vision. Right. We need to give our stakeholders who are taking the risk and funding this change an idea of what we're aiming for. So not detailed plan. We need some kind of roadmap. And so I think splitting out both the platform capability roadmap and how that's going to make the organization more competitive and more agile versus the information products or the data products that we're going to produce to give the right information to the right person to help them make more agile and competitive business decisions. So we end up with two roadmaps. I think is a good way of dealing with it.

Yeah, I think so too. And I think as you're talking about transition versus transformation, that takes time too. And recently had a conversation with somebody about.

This leader really wanted to had a vision for his teams, his data teams to transition to a more continuous integration framework.

As well as to transition from being a bunch of teams based on assets. This group of humans works on this asset and this group of humans works on that asset. He's looking to.

Either figure out if it's through cross training or reteaming to get a single team, have expertise in multiple assets so they can do an end to end data product.

His comment we were talking about, how might you go there? Then he started to say, oh, that's going to be so disruptive to our roadmap. When you said roadmap, I was thinking about that. I said, yeah, it is. He's like, I don't know. And I just kept thinking at what point do you decide that the disruption is worth it and how do you know that? I think software teams went through that same challenge but 15, 20 years ago and data teams are still stuck in. They've got this huge backlog of work. They are projecting this roadmap forward and at any point to say, hey, we're going to have an intentional slowdown in our roadmap while we refactor some things within our the way we work.

Sounds politically risky to them. That was the sense I was getting from him and I get that. I understand why that felt like a scary thing to do.

So they're doing the slow transition. They're doing a very slow transition. They could use a little bit more transformation. I think in that cocoon, little cocoon, just a little bit, I think. But, but that doesn't mean that they're wrong for the way they're going. They're. They're doing fine with the way they're going. They're just not getting the results as fast as they'd hoped. Because again, that roadmap that they have is driving a need to keep delivering value in whatever.

Whatever way. We're delivering value right now and not have a disruption, which I do think. I think when software teams were all going through that together, it was maybe like all the rage. Right. It was probably not so unusual to say, hey, we're going to have a pause in our delivery roadmap to do this change. But now data teams, I don't know that we get that sort of upwelling, that.

Shared experience of doing it all at once.

What do you think that is? Do you think that's because there's such a latent demand to make data more accessible with organization?

Yeah, I think because data touches so many things that mean the backlog is just never ending. You're kind of like never done.

But when you're an apps team, you're never done. Right. If you're creating a CRM application, you're always adding new features. So is there a concept of. Within an application team, there's always really a minimum viable, where it's good enough that you could, you should never stop, but you could stop and it would work for us. In the data world, there's always more data that we can't live without.

I don't know. Yeah, I think I would challenge the idea that a data delivery roadmap shouldn't be disrupted. I think it should for the right reasons. But I don't know if we just don't feel like we have a big enough voice to be able to make that call without political repercussions or what.

Yeah. My view of the roadmap is. It's a guess, right?

Yeah.

And also my experience, and I've been just in yours, actually, it's when we have disruption that we reform and iterate and get better. So we should always be changing the way we work. Sometimes that disruption forces us to accelerate that rate of change.

Absolutely.

As humans, no matter how much we work in an agile way, we hate changing things that seem to be working for us. That's oxymoron. Right. It's an anti pattern. We think, oh, that's been working. Why would I touch it? As soon as I touch it, I'm going to break it, but that's hard, right when the organization's had a guess of where they're going to be, they tend to want to meet that plan.

In terms of scaling, that's one I've really struggled with. And I think from, you know, you've probably got a lot more experience scaling out data teams in an agile way than I have. But right now my thinking is why is moving from one team to two team of humans, by the way. Love that, love that fact that you use the word humans, not resources. Thank you.

When we have one group of humans working and we want to add another group.

Going from 1 to 2 seems to be semi okay. Going from 2 to 5 seems to be a complete nightmare. And I have the perception that in the apps world, in the apps development world, it seems easier. And my hypothesis is because they have more capability to isolate the work each team's doing so they don't conflict or collide. And with us we tend to have some core concepts like customer and product and order and location. Everything's bound to. So we're constantly colliding both on the platform and the things we're using and the data that we're having to leverage to do our work. But what's your view? Is scaling an agile data team harder than scaling agile apps teams?

Yeah, I do think it is for the reason you're talking about. I think.

We'Ve got these monolithic architectures and this touches that, touches that, touches that, touches that. And so it's hard not to step on each other if you're not really tightly controlling who's working where.

I think there we also have some real specialization of skills.

So let's say you have an organization that has a mix of data assets and some are homegrown. And so the skills to work on that homegrown are going to be the development skills, the development language and the database skills and the knowledge about that homegrown thing that's very specialized. And then you have something that's out of the box and you've got somebody, you know, a product purchased off the shelf that has then been customized. So then you have that different specialized skill in that product.

I think that I'm seeing that as a challenge to scaling because scaling, I guess what I'll say is ideally scaling means you have more teams that have end to end capabilities to work within the sphere of influence of your organization versus more asset based teams. And so there it is, a big leap. For the longest time it was a big leap to go from.

Instead of having a development team and a testing team to merge them into one. Right. Like to cross train and to cross skill. Well, even just to get them to be in the same organization was a big challenge. And then, you know, lots of work done to help teaching developers how to test and testers how to develop. And one organization I know recently ensured that all their offshore QE resources come with a development background. Right. So they were hiring those T shaped people.

And that's really cool, that's a really good step forward. But then the next step is, okay, now you've got everybody who have skills on one asset and can test and develop and do analysis and do all those things that a team needs to do on that asset. Well now we want to go across multiple assets and they can be very, very different, but all still part of the same ecosystem and the same value chain to get to the, to the end customer. And so.

It'S a lot of time in cross training, right. Or it could be a lot of disruption. If you have asset 1, asset 2, asset 3. And the skill sets on each of those teams is pretty different, the knowledge bases is pretty different. You can disrupt all three teams and then create three new teams that each have a third of the old teams which then those humans have to re team, which is a.

Not scientific, you can't force it. Right. It's not a, it's not a puzzle you put together and you're done. It takes time. Or you can do cross training across the teams, but that also takes time. So just scaling from that sen.

Is hard. I think the other thing gets back to.

The continuous integration capabilities.

If you have several teams that have learned how to, and have built out their infrastructure so that they can check in little bits of code at a time and run quick little regression tests and sort of sniff test the code before it gets integrated even further into the next higher environment, then you've got a lot of safety around, lots of people working on it. But if you have a situation where the team is used to really only checking in code a matter of days or even weeks, there's not much, there's a lot of risk. And we see that where teams try to take.

Maybe the cultural change of Agile, but they don't have the technical enablers of it, and then somebody steps on somebody else and it's just proof that this agile thing doesn't work because when they were all on one team they could coordinate easier.

About.

Who was working on what and they could have a separation of focus, but they can't. It's harder to do that now without the right enabling technologies.

Yeah. So I think if I was thinking through as you were talking about that, and so if I kind of articulate it back and the way I think about it all of a sudden is so we have skills we need in terms of the way we work. So that's your example of not a separate testing team. Testers are part of the team. They have better skills at testing, but they're helping the rest of the members test better. And then we have technology skills where we're trying to use the same techniques and patterns on our technology platform. So, you know, we're checking in code every half an hour, doing over five days. We're ideally using the same components.

Ideally we. And enhancing those. So if we take all those in a nirvana world as a given, we still start with a problem of that variety of data, that subject. So I kind of, I suppose I'd liken it to. Would we take an application team that was building Netflix and then expect them to build Slack without blinking? It's a subject matter area.

Yes.

It's the same techniques, but it's a whole new area to learn. And when we move from data about customer to data about product, to doing a forecast to those kind of things, we're asking those teams to make that shift. But somehow we forget that and we just expect them to pick up that new data. Understand, because it's just data.

Right.

And I think that's one of the challenges we have. I think one of the other ones is in the application world for Agile, there seems to be a lot more content we can leverage.

It seems to be a lot more published, a lot more available to find and understand.

So, you know, if you think about Scrum, you know, there's so much cool content out there that you can pick up and understand Scrum and reuse it and learn it. And with app development, you know, if we talk about React versus Angular and all that kind of stuff. Right. There's lots and lots of content and patterns and stuff that you can do to figure that out. When we talk about customer journey mapping, again, there's lots of stuff out there on what it is and how you do it. When we look at the data world, it's a bit dire.

Right.

It seems to be nowhere near the content or the quality of content or the content being as open as our application brethren. So I wonder if that's part of the reason that it makes it harder for us at the moment.

I think so. I think that's very valid.

When I do have a good example and I can use that to explain a concept to.

To a team who's new to Agile or they've been trying to do agile and it has been not going well because they don't really understand the why behind it. The examples are where their faces light up and that's where they start to get it. I have found over and over, you give one data team an example of a concept, but with using real data, and data geeks are smart people, we can take that one concept and explode it and apply it to our worlds, but without those examples, without those patterns to follow, like you said, there are articles that have been written. Even when I first got into bringing this together.

10 years ago, there were articles that had been written about applying agile to data, but they were just words.

I was looking for something that had a table in it that was going to show me some data. For example, you go from this to that. That is what I really needed to see to help wrap my mind around it. But it's the examples and when I spent a lot of time doing training for a lot of different organizations and one of the things I found is that if going into, before engaging with a group in the training room, if I could get with one of the leaders in the organization, not necessarily the leaders, but one of the team members, and we could come up with an example to anchor on for the Agile training, it was much better. I started investing some time up front in getting a concept of their world, what their business is like. And what's an example? Just an example. We could write a couple user stories around an example, we could write some gherkin around for the tests, that kind of thing, then people were like, oh, I get it, and then they move forward. But if I stood up there and gave them a lot of words before I got to an example, there's a lot of skepticism, right. And a lot of people not really listening. And so I agree with you. I think there's a dearth of content, really. I tried to do my part and change that when I had my own company.

Since I joined a company as an employee a couple years ago and I have not been doing any blogging since then and I'm feeling pretty guilty about that. But I'm glad to see you've picked it up.

I'll tell you what, I've done a lot less blogging than I've done in the past. But yeah, it's interesting. I was like, when you made that change, I was like, yes, she's not going to be so busy out finding work and doing work. We're going to see nothing but blogs.

Coming out of her.

And I can pick up all this knowledge in such an easy way. And it didn't happen. I was very disappointed.

I do want to say part of that was I had some things going on with my family that I needed to take care of, so I just sort of. And then I just got comfortable. I'll be honest with you, I've just gotten kind of comfortable. I work with 14 data teams and they're wonderful, wonderful humans. And I am thoroughly enjoying being here for the long game. And I just kind of haven't picked my head up from the cocoon that I'm in.

And 14 data teams, that's just ridiculous, right? It's a lot for the ability for you to scale to that level is just awesome. I think for me, one of the interesting things people always say, and especially when I go and meet other people in an agile area, agile scrum masters or coaches, and I say, look, I help teams adopt an agile way of working, but I only work with data and analytics teams. And they're like, well, why you should be able to work with any team? And I say, yeah, I probably could. But for me it's about those examples that I'm long enough with enough teams now that I feel when I'm with a new team and they go, we're stuck on this problem, I can say, well, actually I've seen the team do this and it's work for them. I've seen another team do this that's worked for them and I've seen a third team do this and it's worked for them. Why don't you just experiment with one of those and see what happens? And you're talking their language, you're talking in the language of data, you know, the complexity and the. And they seem to be more open because you're giving them examples that are based on real experience. And that makes it a lot easier to help a team adopt some of those things or experiment with them?

Absolutely.

To close out here, looking at time, you've been doing a lot longer than I have.

Has it got easier? Do you believe if you look at applying agile, an agile way of working with a data team 10 years ago versus what you're seeing now, are we like before, like the application teams? Right. We're getting over that hump.

Yeah, I do think so. I think so, because that's a great question. I'm thinking back on like my first few clients.

It'S easier to convince people to give it a try because there are more examples in the marketplace. And I think also.

A lot of my career as a consultant was fueled by following agile coaches, who were very good agile coaches in many cases, but they would go in and work with a client and they'd make great progress on everywhere but the data teams. And then I would get called in to kind of can you just get them over the finish line and then we're done with this project. Right. Like I was sort of. And so I think.

Early in my, in that venture I was. Because I was coming in after the teams were already worn out from dealing with an agile coach that didn't understand their data world. The resistance was greater. And so I think early in that, in having my company, I was dealing with people who were tired and worn out and a little pissed off. And so that was harder.

Later in that time of having my own company, I was coming into teams, were even maybe.

One of the last clients I had, the data team wanted to lead the agile transition for the, for the company. And so I think that shifted. I think.

I think, yeah, I don't know. It's a really good question. I think our technologies, I think the. Well, certainly with things like data science and advanced analytics that is really, really well suited for an iterative, incremental approach. And so I think as people who are working on maybe older approaches to data management see the new cool stuff, also using Agile, then that was sort of an inspiration. But it can also be frustrating because there's a difference between hand coding something in Python and using Informatica. Right. There's, there's pros and cons for each, but from an agility standpoint, the Python is going to be more agile by nature.

So I do think that I'm definitely still enjoying what I'm doing. I think there's still a huge need.

For this intersection. I'm really glad you're still out there in the market and that you're doing what you're doing because it's much needed.

Yeah, well, I think we're. I'm still loving it and I just, I just get so find it so awesome when, you know, I'm working with a team and they've just found something that works for them and they just love it. They won't go back.

Yep.

It's like that's that success. It is. And you know, data is the new sunshine, right?

Yeah.

We're in a hot market and hopefully it stays like that for a little bit longer.

Yeah.

So, hey, look, thanks very much for your time. Really, really appreciate it. It's been great to catch up.

It's been great. So fun to see you again.

Yeah. I think we'll probably come back in six months or a year with another subject and do it again.

That'd be lovely. I'd love it. That'd be great.

And that Data Magicians was another agile data podcast. If you'd like to learn more on applying an agile way of working to your data and analytics, head over to agiledata IO.