AgileData #31 - scaling data teams - Tammy Leahy

Welcome to the Agile Data Podcast, where we talk about the merging of agile and data ways of working in a simply magical way. Welcome to the Agile Data Podcast. I'm Shane Gibson.

And I'm Tammy Leahy.

Hey, Tammy, how's it going? Great to have you on the show. Been trying to get you onto the podcast for quite a while now. So before we rip off and talk about all the cool stuff you've done with your team scaling them out, Why don't you tell the audience a little bit of background about yourself?

Sure. As instructed, I haven't prepped for this, so it might be a bit meandering. I've only been working with data teams for about 4-ish years, and I come from an accounting and finance background. So, I'm deeply technical at my core, and my kind of what I came through university at, and I was very passionate when I was going through university about making the world a better place. I started off wanting to be a translator, so I did linguistics, Japanese, Chinese, and my parents said, "No, no, no." get a backup because that doesn't pay. You won't be able to pay your bills with that. So, I became an accountant. And as part of that, did accounting and philosophy. Saw it as a way of viewing the world, same as economics, same as music, languages, everything else. So, my whole thing is about thinking about the world in patterns, whatever your chosen language of choice is. So, I think that's why I've perhaps ended up in this space because I do see patterns and ways of making teams work well together because that's one of the things I'm really passionate about is how do you get teams people enjoying their work and just loving what they're doing and doing good shit, really, you know, like getting to awesome and feeling awesome about themselves. So, yes, I think I've been in this, in this role, in this team, yeah, like I said, for about 4 years. We started off with about 8 of us. We're up to, I'm not sure quite how many sometimes because we work with a few partners.

All right, so if you had to pick how many people are working in the data analytics space in your teams right now, what, what, from 8, what would the number be this week, do you reckon?

Um, so in terms of in our direct team, uh, it's around the 22 to 25 mark. We're recruiting at the moment, so it's probably going to be up around the 30 mark in about 3 months. And then we've also got kind of, we call our data co-op. It's a group of vendors that we work quite closely with, and there's about 19 of them in working with us at the moment on quite a big piece of work. And then in the wider community, we've got a decent whack of people who are just going there. I think there's the thing about the community is a sense of belonging with people, and it's really cool how people in the community are starting to talk about it as a place that they belong in or belong to. So that community, at the moment, we've got about 100 and something over 100 active participants. And I don't know how many inactive because we've got many, many more people actually using the tools that we put out there and that we kind of coach on and provide learning on. Yeah, so I don't like to think about the numbers because it's not about the numbers for me. It's about the outcomes that we're trying to achieve with what we're doing.

Cool. So, so many questions we could go off on. So let's, let's start back in the beginning of it. So, When we talk about numbers, right, we increase the numbers of people on our teams doing the work because we want to scale the work to be done, right? And we know that adding more people makes things harder. Every time we add new people to a team of people that are working well, we lose some velocity, right? We have to reform and change the way we work slightly. So, when you had 8 people right at the beginning, kind of think back to then, what was the team structure? What was the team topology?

Right. So, when we first started off, it was just before we'd worked with you, actually, thinking right back then. I actually found the picture on my phone of the first team forming, the first team self-selection we did. There, it was a combination of a BI team and a kind of a business performance reporting team. So, it started off as two silos, really. And we had a really strong BI team lead who is actually our lead engineer now, and then me. Who'd come from all this commercial context and thinking about the value we could deliver. So, the team structure was still a bit of a— it was absolutely suboptimal, throwing shit over a wall at each other and expecting someone to understand what you meant. So, that's why we went down the path that we went down to try and get a bit more closer collaboration and work amongst the teams. The analysts understood what the engineers were doing or the BI developers were doing, and the developers understood what the analysts meant when they said, "Oh, but the user wants blah, blah, blah better." So, the structure that we're— and sorry, when we say structure, I talk about structure as like an operating model or a way of working because I honestly believe that HR lines become almost meaningless when you're actually working in a properly collaborative way. And I think of myself as a person that wears multiple hats, which represent the functions I fill for my team. So, I think of— yeah, sorry, that's going on a tangent. I'll come back to that in a minute. But, yeah, so, back to the structure of the team. We ended up going from this kind of two functional silos, if you like, to more of a squad-based approach or team-based approach. Small teams of 4, typically 2 analysts, 2 engineers, or at the time they were still called BI developers. They hadn't yet learned the new tools we were about to start implementing. And then at that time, we didn't have that concept of a product owner. So, between me and the lead engineer, we were multi-hatting everything. So, I was the people lead for everybody in the team, and that was intentional. When they did the restructure to bring our 2 teams together, I spoke to the tech leads at the time, and their view was that they wanted nothing to do with pastoral care. That was not their strength. Their strength was in mentoring and coaching and thought leadership. And my strength was clearly in people side and coaching. So, yeah, so basically went completely fat, which was okay when you're at like 8 to 10 people tops. So, that was one hat I would wear. I would also wear the product owner hat or product manager hat, depending on how you define it, for the platform, the data products we're delivering as a proxy because we didn't have the business engaged in that way yet. Yeah, our lead engineer was covering everything from the architecture to some of the building of the platform and the infrastructure. So, you know, lots of multi-hatting, not just double-hatting. At one point, I kind of figured out, even in the last year, I've worn up to 5 functional hats to make sure things get to where they need to get to.

So, that was one of the first things we did though, right? Was we worked out the hats that needed to be worn, and even though you ended up wearing a hell of a lot of them, on your head at multiple times. We always talked about them as different hats. You know, we talked about the pastoral care, right, looking after people, making sure they were safe, making sure that they could have a career, make sure that this change, as horrible or not as horrible, but as disruptive as it was, you know, they were as comfortable as they could be with that change. And we talked about the hat of work to be done, you know, the prioritization and that kind of stuff. And we talked about the hats of, you know, designing or managing the work that is being done and leading that and mentoring and coaching. So, we always talked about multiple hats. In terms of using that framework to kind of start to drive the skills or the roles that are required within the teams, how much of that did you think played a success as you scaled out to more than 8, right? As you started to add more and more people to the wider team and scale out that work to be done, Did those thinking about it in terms of hats and which hat you're going to scale, did that help or not?

Yeah, it has actually, because I think for us, it was, it didn't feel, it didn't feel like we were scaling for a long time. So, it didn't really feel like we were at a painful point until 6 months ago, really. And I think as we, and we never approached it as a, oh, we need another role or another function because We need another, you know, it was never just because. It was because we noticed some, in some kind of feedback loop, either through employee engagement surveys or through just talking to people one-to-one, that we're noticing some part of our system not working the way it needed to work. And then you try and diagnose that and you get down to the root of the problem. It would be because we were wearing too many hats. And so we'd then say, well, which hat's the problem? It's that one. We need help with that one. Let's find someone who can help us with that. Particular piece of our system. So, for example, when I was running, up until about a year ago, I had the kind of, I hate the word evangelist or advocate for data, but I was running that function of helping the business understand what's possible, you know, thinking, pushing them to try and dream bigger and use bigger things with data. I was wearing that hat and then we had our lead analyst come on and he picked up that from me, which then meant I could focus a little bit more on stakeholder engagement. But by then, demand had spiked. So, there was a whole lot more delivery management type work, helping people people prioritize amongst themselves what work the squads are going to do because you had conflicting product owners. Then our delivery practices lead came on and she picked up some of that, as well as a whole lot of the people coaching. But it turns out that my capacity for work perhaps is a little bit too big. And so, what we've ended up figuring out is actually, we're still constrained. And if we look at our last, I want to say experiment, when we brought on a whole new squad, which I would not ever recommend. At least not in the way we did it, because it was a terrible— the feedback was certainly telling us it was a bad experiment. We clearly weren't in a position to bring them on because at that time we didn't yet have that extra person in our support crew to help with the onboarding, with the pastoral care. Yeah, so like I said, we focused on what are the— what is the feedback telling us? What is the team telling us? And what is our system telling us we need to do? Where are the problems, then how do we need, what hat needs fixing? And so, yeah, I can go into so much detail on this. We actually did an exercise this year when, it was earlier this year when we were looking at, we just, there was, it just, the system felt like it was at breaking point. The pressure was massive. And so, we sat down as Support Crew and said, let's do a functional mapping. And I'm using air quotes because it's effectively a hat mapping. What are all the things you're doing? So, we did it in person. There were 4 of us. There's only 4 of us still in Support Crew with this practice. And we got different colored Post-it notes each. We had a big table and everyone had to write on their color Post-it note all the things they did daily, weekly, monthly, quarterly, whatever it was for the team. So, we did that, wrote them all down. Turns out I was still wearing way too many hats because when you wear lots of hats for quite a long time, it's difficult to recognize that you keep putting one back on and you're not letting it go to the person who should be wearing it. So, we did all of that. Then we took everything and kind of grouped them up into the I'm going to keep using the word hats, into the hat buckets. And then when we looked at the different colors and the different hat buckets, it was very clear where a lot of us were just chipping in because something needed to happen, but we weren't focusing on it well enough. And it was really clear where someone was doing way more, like their cognitive load was massive because you could see their color ticket across everything. And so that was a pretty cool visual exercise for us to do to say, for ourselves as like a— we call ourselves support crew. We're like the tech leads, the people lead, that's us. For us to figure out what extra hat do we need to bring on now to help us, like, what thing needs filling because we've got problems in our system. And we had some, like, in my view, pretty, pretty awful problems because, yeah, this onboarding, this new squad, we got some awesome feedback from that.

So, So you call it support crew, I call it coaching team, right? So to be clear, it's a group of people who are sitting out of the squads doing the work to help figure out what needs to happen next from a scaling point of view, right? Where the blockages are, where the problems are, where squads need help. So the ability to sit back and observe and then work with them from a retro point of view to go, okay, we probably need to change this. So I just want to go back to that. That feedback loop. So, one of the patterns that I saw you do really, really well, and I also saw Jan Shepherd do it when she's been on the podcast before with her team, was right at the beginning, you spent a shitload of time every week talking to your team one-on-one. And so, my theory, right, my hypothesis watching that a couple of times is at the beginning, that allows you to get very quick feedback from every individual team member about how you need to change, how you need to guide or coach on the way we're working. And without that, you're going to go down a path and then not get any feedback and hit a problem. Is that true, right, that that actual, you know, it was what, half an hour, an hour every week with every person at right at the beginning to get that constant, um, you know, kind of direction change where we have a problem that we may need to look, keep an eye on, and then maybe need to help mitigate later on.

Yeah, so I think that is true. I did put— and that's just one of my practices— is people need time, they need to be heard, they need to be recognized, they need to be seen by the others they work with. So yeah, I did spend quite a lot of time with them, and I still— until our delivery practices lead came on and picked up some of that, that more pastoral care element from me, I was still doing that at a time when I had 24 reports. I was doing half-hourly to hourly one-to-ones. At that stage, it was between weekly and three-weekly, and that was dictated by the person. So, I didn't say, "We're going to meet three-weekly." It was a, "What cadence do you want to meet on? How often do you want to chat?" And sometimes, if people have more going on in their personal life or more going on at work, or you'd have a more frequent cadence, you'd have longer cadences where, you know, people don't need to talk as much. But absolutely, I think, because you'd have your retros and, you know, when teams are learning this stuff and people are getting there, sometimes they don't always bring up the the things that are really the problem. Like, there's a symptom they'll see, and they'll tell you about it at a retro. And then when you have a conversation, the most effective way this was done, actually, was I found if I have booked in the one-to-ones with the individuals all in the same squad on the same day, you start to get a really good feel. Like, at the end of the day, you just have this vibe and this feeling for what's going on in that team to know where issues are. And sometimes people actually say things that other people have on their minds. They say, "Oh, did you know so-and-so is thinking this?" And you're like, "Oh, shit, no, I didn't know that. And actually, they haven't said that to me. I wonder why. Like, is there something about openness?" And then I'd sort of find a way to broach a topic, you know, in a very open kind of question-type way to then open that conversation with so-and-so, you know. So, that was really helpful. And actually, that's one of the things I think I get a little bit uncomfortable about with scaling because how do you keep that quick feedback loop? In a really authentic way because I actually miss that. The further away from that I become, the more I miss it because that's such a deep connection to the team and the people that not having that, yeah. And again, maybe it's me, I just need to let the hat go more, but I still at least do quarterly meetings with everyone in the team. And often you'll get people message you saying, hey, can I have a quick chat? So sometimes you don't even have a regular meeting set up, but you just talk to people anyway. Yes, that's absolutely a feedback loop.

Yeah, I think for me, it's, you know, I've got this thing coming out from the No-Nonsense Agile podcast that I do, that there's a difference between managers and leaders. And, you know, for me, I can now articulate a pattern where the manager focuses on the work to be done, the leaders focus on the goals that the team should be working towards, and then helping the team be successful. And so, that pastoral care, right, that how's the team feeling, how's, you know, because we lose a lot of that in agile. We go back to becoming focused on the factory, right, focused on the work and not focused on the people. So, we lose a lot of stuff if we don't manage it, if we don't put hats in place purposely to do that pastoral care.

But I think the other thing that you have to have as well, because we noticed it when, because we'd only been working with you for what, 9 months, and then we went into that first lockdown. And I think if we hadn't done that work before we went into that first lockdown, we would have been been in deep doo-doo during the lockdown, like remote working. So it actually helped us a lot to be in that frame. But a few of the other bits, it's not just pastoral care, it's team coaching. So it's team care. So how do you care for— you care for the human being as a human being. How do you care for the human being as part of a collective of some description? Then how do you care for your system? So it's still not about the work to be done, but it's how your whole— how the people interact together. So that social system, but also the things that help that system function. So the way I see support crews, our function is to make sure people are cared for, teams are cared for, and our system can sustain itself almost without us. Like, that's the ultimate goal, is that we could step right away and this thing continues to work because the people in it are leaders in and of themselves. Like, yeah.

So When we first started scaling, we had a really good conversation about, okay, when we're gonna go to another squad, we had some choices, right? And so, one choice was bring in a whole new squad, and the other choice was split the current squad in half and bring in new members to both halves. And, yeah, pros and cons on both of those. And from memory, we decided split the squad, right? Oh my God, that's so long ago.

I wish I'd remembered that.

That was so long ago.

I wish I'd remembered that 6 months ago.

Now, so that's my question, right? So, 6 months ago, it sounds like you experimented with landing in a whole new squad.

Yep.

Talk to me about it. What went wrong? What went well? What went wrong?

What went well? I suppose because I care so much about the people, I feel like it went terribly badly because we got some pretty strong feedback from the individuals that joined that they didn't feel supported enough. They didn't feel well enough cared for when they joined. So, we'd set up, the way we'd set it up was we had a whole new squad start basically on the same day. So, what was good about that was they had almost like a cohort they were part of. So, it's like, you know, I come from a Big Four accounting background. I joined as part of my first 2 weeks was spent in a graduate training program, you know, and you end up with a cohort and that creates a sense of belonging. So, in that sense, they were a really strong unit in that sense. They were almost like us against the machine kind of thing. I think it was a bit, so that's where it was bad was us against the machine, but it was good in the sense that they were a cohort. So, that was good. I think it was good in the sense that you didn't end up with anyone having potentially useful new views. Because I'm very strong on that idea of I want our existing team to be challenged as much as possible in as nice a possible way with how we're doing things because there's always a better way, right? So, it was useful having a team come in completely cold and completely new together without almost being indoctrinated, for want of a better word. Not that we indoctrinate people, but, you know, like, they were, they were in a position where they could completely define some new practices, think of ways that are different, which equally is bad in some respects. Yeah, so I suppose those were the good things. The not so good things were that, because we're full remote, we were having to onboard them remotely because it was during lockdowns. Just that connection with the rest of the team was really difficult to establish because there was no incumbent no existing person to sort of start the squad. So, we had buddies in place, we had the meeting with them regularly, they had a really close connection with our lead engineer, but it still wasn't quite good enough, you know, it still didn't quite work to remove that sense of isolation. We had a really awesome product owner working with them, and so that helped to an extent. She connected them quite well with the business, and, but yeah, it did feel a bit, isolated. Yeah, and then I think, I think just the feelings, like our engagement survey. So we onboarded the squad around October, November. We had an engagement survey run in December. And oh, it was just like, because you can look at employee lifecycle and the team's big enough that you've got, you can keep it anonymous enough within the cohorts. But the new joiners cohort, which wasn't just that squad at the time, there were 2 other people in the new joiners cohort, had an engagement score of 6. Compared to the rest of the team where the engagement score was sitting at like 8.5 or 8.7. So, you know, that's, that is, if that's not telling you something. So they were saying things, but, you know, I wasn't doing one-to-ones with them at this point. I'd handed off the people lead hat to someone else. And so, like, I feel like that's part of this concern I've got about having the pastoral care hat handed off because the feedback I felt wasn't quick enough. Or maybe we, heard it, but we were too busy with fighting fires because of rapid scaling elsewhere that we didn't hear it and act on it fast enough. But yeah, so many learnings have come from that. And I think one of the things that we did was the second we saw some of this feedback coming through and we saw some of the messages coming through from that squad, we just basically, I set up one-to-ones with all of them and said, look, I'm really sorry, this is a shit experience. This is absolutely not what we'd hoped. You would have had coming in and just worked with them individually to start with to understand where they're at, what their concerns were. I think a lot of it came down to, really, it was a connection thing. Like, if you bubble it all down, it's a connection thing. And when you're remote, and this squad was one of our first full remote, like we had guys down south, guys up north, like everywhere. That's been a real challenge is how you get teams onboarded and connected like properly connected, like a real human connection. That was a big problem, I think, with that stuff.

So, yeah. So, one of the things I say when I start coaching is let's remove as many of the uncertainties as possible, right? So, we have a— Oh, but you know me, Jane.

I go for chaos.

I know. I know. So, but you did bring, you know, you said you brought in an experienced product owner, right? So, you remove the uncertainty of a product owner who hadn't worked in this way before.

Well, no, actually. So, yeah, no. She wasn't experienced, but she was a good product owner. So, she invested a lot into, we coached her quite a lot, and she invested a lot into understanding what was required. I think what the squad found difficult was she wasn't a data person. So, she'd come from a research and insights background and was amazing because at least she had that and she could connect the dots. But she was really well connected with the business and understood the business needs. So, in that sense, very strong.

But had she been a product owner with any of the other squads? No.

Okay.

So, do you think if you had brought in a product owner who'd been a product owner with the other squads before that you would have reduced the uncertainty for the new team in some way?

Yeah, absolutely. Absolutely. So, when we diagnosed this, when we boiled it right down, we said, we can see 3 key issues here. So, the data product owners that we— so, the way we operate with our data product owners is they're business people who have day jobs who commit to us at least, you know, 20% to 50% of their time to support the squad depending on how intensive and how new the stuff is they're working on. And that's a pretty hard commitment. Like, if they can't commit that time to us, then it almost becomes a, well, we're going to put someone else in, and then you might not get exactly what you want because this is just how it's going to work. So, what we found with that model is that in a world where there's lots of demand on everybody's time, that becomes a real— both it's a learning curve that these people, if they're committed to it, they invest their time and effort in, and mostly they're just discretionary time and effort in alongside their day jobs. And it becomes like one of the feedback things we had from this product owner was, I really enjoyed doing it, I've learned so much, but it's so, it's such hard work. Because she had to get her head around not just, she had to get her head around data and what it means and some of the decisions she's got to make, which is different than, you know, like a product owner who's looking at a customer offering through a product or a plan. It's very different. In data land because some of the decisions you have to make are in the detail of, "Oh, I'm seeing this edge case of 5%. Do I care about it?" You know, like, that's different than, "Does the button I've got to click on this page do what I need?" You know, it's a different problem. So, that was one of the things was the burden that, oh, I feel like it's a burden, but it's an opportunity equally that was on these product owners is quite high. And she articulated that really well in the sense of it's a great opportunity, but equally it's hard work. And that that makes it more difficult. The second thing we noticed really quickly was our support crew, which is our coaching crew in your parlance. There was, at that time, was 3 of us. We hadn't fully onboarded the 4th. The 4th had come on, and partway through, we were transitioning this people lead hat. So, by the time the issues, you know, the shit was really hitting the fan, the people needed to transition. So, yeah, total chaos. You know me. So, that was one of the issues. Clearly, we did not think about this well enough. We just barreled into it. We were too busy on onboarding this whole other program work that we didn't think about it well enough. And then the last one was really around just having a better remote experience. So this idea of, but it comes down to that thing of having enough time to think about it. So at the heart of it, it was support crew wasn't well enough. I don't want to use the word resourced, but we didn't put enough priority on bringing this new squad in because it's just a matter of priorities if you're constrained in a constrained world. And from a data product owner perspective, yeah, let's do something to have some maybe more dedicated data product owners who understand this world better, can actually help coach these teams. That was the other thing. We didn't have any coaches in because we didn't have a team facilitator. We didn't have an agile coach in. So, between me and the two other guys in support crew, we were wearing those hats amongst everything else. So, I can see so many opportunities where had we put some safety measures in place, you know, better coaching, like at least a team facilitator or a Scrum lead or a coach, get someone like that in, actually have that people lead stuff solid and settled, prioritize, you know, like there's so many things we could have done better. And actually coming from that, we diagnosed those problems. We went, we dug, dug, dug until we said, well, I think this is the root of the problem. That has now resulted in us proposing a different model, but it's not different. It's just an evolution of what we were working on before.

Yeah, so I think you described a good case of perfect storm, right? So, you know, the coaching team was going through a transition, they weren't stable anymore because they were changing at the same time, you brought in product owners who hadn't done it before and a whole new team. So everything was changing. So it makes me think, and it's a pattern that I haven't tried before, but actually, if we could map out some of the core practices and then basically give it a chaos score, right? And say, okay, given the amount of change, then right now, we need some either extra focus or we need to not do it or we need to change something, right? Because we know that the chaos score on every line of that practice is red. So therefore, yeah, we might be successful but highly unlikely. So, going back to that though, so you'd already scaled though, right? So, I can't remember, we went from 2 squads to 3, then you said you got to like 20 to 25 internals, 19 co-op partner people.

So, that was all that happened. So, we went from 8 to about 15 over the— so, it was a relatively slow-ish scaling. For the first, I don't know, 18 to 24 months. And then in 6 months, it went from that to boom, this vendor, this whole lot of this big program of work happening with the vendor co-op. Then there's a new squad coming on board. But that slower scaling period, I think we were a lot more intentional about what we were doing because there wasn't as much demand on us. And I think in a way, part of how we've worked has created the demand. So, it's a good problem. It's not a bad problem. But we did actually test a whole lot of different things in that, I'm calling it the gentle scaling period. During it, it didn't feel gentle, but relatively, it's gentle. But, I mean, but even the first bit of joining the teams together, that was a huge mindset shift. So, the very first step of getting the teams to work differently, and like, there's one of the guys who's a senior engineer in one of the squads. At the start, he was like, "This is just micromanagement. I don't see any value in it." But now when he talks about it, he's like, "I can't imagine any different way of working. This makes us safe." So, we use the word safe quite a bit in how we think about things. And we certainly weren't safe through this last not gentle period of scaling. But yeah, so one of the things we did when we brought, when we went from 1 to 2 squads, but also we never really had a platform squad, the squad that built the machine that we fly with our data products, if that makes sense. We kind of had these platform-type engineers embedded within the data product squads. I don't know if you remember that. We kept having these problems of actually we never really quite got to building out the platform components. That we were very just-in-time. We never build anything unless we can see a clear use for it. We never quite got to doing some of the things we really wanted to do to make life better, easier, faster for the people working on the platform. So, we then brought— the first step was to go from 2 squads with sort of a lead engineer and, I don't know what, and kind of a jack-of-all-trades sort of developer sitting outside the squads. Then we brought a couple of people who had a much stronger interest in that sort of much more hardcore technical space. So, if you think about T-Skill spectrum, they're way down the almost cloud and infra type end, brought them in and created, like, the platform squad. And that was great because that allowed us to create some focus on what do people actually need to be effective, because building it while you're trying to drive it or fly it, like, it's painful. And if you haven't got focus as you're doing that, it's even more painful. So that was one of the things we did. So that helped a little bit with the scaling. It meant that we could bring on or change the squads developing and building data products much more safely. So that was one of the first patterns we adopted was to create this platform squad. And the platform squad does not maintain anything that the data product squads build. So if they build it, if they build a data product like a dashboard or a data service or whatever it is, and it breaks or it needs to change, it's their backlog. They have to deal with it and we reserve some of their time to deal with that kind of stuff. The platform squad's really about, you know, like managing the components that everyone else uses, making sure that actually they can do their work effectively. So, we found that those squads, the data product squads are becoming constrained because, I don't know, the clusters we're using are not running fast enough and they're waiting in a queue. Okay, platform squad, figure out how to improve their experience. And over time, that's evolved to talking about now developer experience and how do we think about making the experience of our squads and other data product squads better. So that was one of the first iterations of scaling for us, was bringing in this platform squad and separate data product squads.

Yeah, so one of the things I liked was, you know, if I use the DataOps brand, Definitely one of the earlier teams I worked with that came fully on board with that idea as a data product squad, that theme of you built it, you deploy it, you own it, it breaks, you fix it. And that's always a problem, but that idea of just holding out a percentage of time so they can have the right to fix things, that thing. But what it gave is there was a lot more focus from the teams on delivering things that wouldn't break because they knew they had to fix it. And then the second one was that move towards platform as a product, the idea of a squad who are building a platform, but the key focus was they're building it for the data product teams, right, squads. They're not navel-gazing off and build cool shit. So, there was always that balance of how far ahead could a platform squad be in building the product, how do they get early warning, when there's something that's needed that would take longer than an iteration, so they had to start working earlier. How do they know that they're not over-engineering it, but how do they also get feedback from the other data product squads about would this work for them? So I think I remember, like, one of the ones was the early modeling and lineage and embedding it, the, you know, effectively a bunch of hints in the code that then formed those lineage maps. And that that to and fro. So, how did you get with that? How did you get on with that idea of the platform squad building a product, a platform as a product, the customers being the other data product squads? How do they get early warning? How do they not overbake? How do they make sure it's fit for purpose? How do they get feedback from their customers, which are the other data product squads? Where'd you get to in the end with that?

So, actually, this is where we're trying to, we're experimenting a little bit at the moment with, improving those feedback loops. So, the platform squad itself has ended up growing a little bit, mostly because the number of squads we're supporting, the number of users we're supporting has grown, which indicates that we haven't automated enough to scale effectively enough, but it's one of those fast, you know, it's a problem of to go fast, you have two choices: automate as much as you can upfront so you don't need to bring on more people or bring on more people and fix it later. And we typically do the whole, what's the minimum we can do? So, we brought in, like, basically one more person into that squad to support And I think typically what happens is the team just talks to each other. So, one of the patterns we've got happening now is the senior engineers in the data product squads are almost like the beacons that will talk to the senior engineers in the platform squad, and they'll share, and they'll be like, "Oh, this is not working for me," or, "This is working, but I'm having trouble with this piece of it." So, there's almost like just some chats happening within a Teams channel we've got where people experience problems.

So, that's key though, right, is that the the senior people in the data product squads are not logging tickets for the platform squad about what they need. They're having a conversation with them to say, hey, this is where we think we're going, these are the things we look like we're struggling with. So, it's a conversation first.

Yeah, well, I mean, so we've got, I mean, we have a pretty long-running backlog for platform squad. It's got so much on it that we'll never ever get to all of it because it's that vision of what would a perfect thing be. But actually what is the next most important thing for us to do? And that's how we tend to approach it. But I mean, if you would talk to some of those guys in the data product squads, they would say that we've never got onto stuff they wanted quickly enough. You know, like, it's, you can never do enough, but that's okay because there's always more to do. But I think at the very beginning, what it ended up being was in retros, the guys would complain or, you know, what didn't go so well would be, oh, we couldn't run this thing because the platform couldn't do X, Y, Z. So, to start with, That was some of the feedback loop was through the retros. But then over time, as we've kind of created this more collaborative practice amongst that group of people who are interested in the engineering world, as they've got closer and as we've seen leaders emerge more and, you know, someone's got a really strong interest in testing or someone's got a really strong interest in improving how we deploy, you know, then they'll talk to each other more because that's just happening. And actually, one of the people that joined the team as part of that new squad, not even an engineer, just started a lunch and learn session randomly. He was like, hey, let's just get together and talk about stuff. So, you know, I think it's really cool to have the openness within the team to say, let's just talk about stuff.

But, but that's a good pattern, right? So you didn't just pick up the Spotify model and draw the lines in the woods. You didn't just pick up the SAFE piece of crap and try and force that to happen. You, what you did was you knew that those things may organically grow when they have value, and you watched for it. And then as people started to create those groups outside of a squad, you encouraged it, right? And if there was value, they'd keep doing it. And if there was no value, they'd stop doing it. And that's okay. We don't need to put these artificial lines in and try and drive it because we think it's valuable. The teams will do the things that make themselves successful, and we just need to observe, hint, and sometimes guide, but never do, right?

Well, I think sometimes you don't even need to hint, honestly, because these are smart people and they're passionate about their work and you give them freedom, they will find awesome stuff that you could never have even thought about. So, yeah, I—

So, how, when you started to scale, when you went through that mass, that uber scaling, how many of your practices and patterns were kind of stable, were put together in a way that you could actually teach somebody new coming into them, or were they still relatively ad hoc and that was one of the problems?

Yeah, there are two steps in that kind of faster scaling, the not gentle scaling piece. The first step was onboarding this whole other program of work with 15, started with 15 developers and then 19 all up. That one, we ran a pilot, so we kept ourselves slightly safer. We ran a pilot because the first time we were getting We were allowing, I say allowing, we were getting not our people into our platforms in a way that, you know, you could do some pretty bad damage if you didn't know what you were doing. So, we wanted to make sure there were some safety measures in place. So, we tested it as part of that pilot process with one additional squad, and we were really open with them about how we were working, what we were doing. They gave us great feedback about what they needed to see, what wasn't in place. So, coming out of that, we've got this thing which we call Docsite, which is effectively it's just a repo that's been surfaced through a webpage. You can only access it when you're inside our network. But that's got it, like our data catalog, it's got our ways of working, it's got our principles and standards. And so, what we did was, I say we, what our team did was they built this standards and practices section. There's like a BA, like a BAU or a user guide section. So, when you're onboarding, follow this guide to get yourself set up. If you're new, here's the standards and practices, these are the principles. The philosophy is that you don't want to die in documentation, but you've got to have just enough that people can understand and at least they know when they have to ask a question. So, that was really helpful running that pilot and understanding that, oh, shit, we need to document a little bit more of how we work and, you know, in the technical practices that people understand it. Then as we brought on more people, so that was in place. Then as we brought on more people into that program of work, we very quickly figured out that one of the patterns we adopted was this concept of a data translator. So, something that we realized quite early on with our squads was don't commit to a sprint and refine the work you're going to do, but you've not really done any of the thinking around the modeling or the design work. That's okay if you've got a product owner who's really close working with you, you've got people who understand the business, they can actually get an outcome within an iteration. When you're working in a place where you've got people who don't understand your business, you kind of need to provide them with the right level of support. To be able to get the data products out in a timely way because, you know, this is a project, it's constrained with time, money, all the rest of it. So we introduced this concept of our data translators and they were working ahead of those squads to define the models, define, working as that interface, I suppose, between the business means or the business owners and these squads. And then one of the new patterns we had to implement as part of that scaling, this was as we were bringing on, we went from one squad in that program to, I don't know how many we're up to now, 4, 3 engineering and 1 kind of more vis squad. And this is also different. The way that is working is different than the way our internal squads work because our internal squads don't split between dev, vis, none of that. Our internal squads end-to-end. But this was more about what can we reasonably expect of people coming into this world. Anyway, so we implemented this new thing called a data design forum, which to start with was really just a way for us to teach people how to model better. And for an opportunity for people to sense-check models. Because part of what we do is we document in a coming soon section the thing that's going to be built. And so we standardized a template around that. High-level headings, why are you doing this thing? What is the user getting out of it? Define your tables, transformations in enough detail that someone can pick it up and work with it, someone who's reasonably qualified, right? We don't want to turn people into code monkeys, And so, that pattern evolved because we found that if we weren't providing an opportunity for people to challenge each other's work or to get someone else who might have seen that domain before, like that subject area before, to say, "Oh, hey, you haven't thought about this problem with it," we were getting lots of rework coming back through these squads. And when you're in a time-constrained, cost-constrained space, rework's not great. So, we implemented that. So, it's almost like a It's not an Arc forum, it's a data design forum. So, we implemented that, and that helped a lot. But yeah, that—

Yeah. So, you know, that idea that I kind of just had around uncertainty mapping when you start to scale, effectively, you've applied that to your iterations, right? You've gone and said, okay, if we— and it'd be interesting to see whether you could codify it, right? Here's some lenses, and if we could score the uncertainty in those lenses from 1 to 3 or 1 to 5, you know, when we know it's a domain that we have no expertise in, our product owner's new, you know, is not data-savvy as much as some others, you know, a whole lot of things, you go, our level of uncertainty is high. So, we want to do some work early, you know, to reduce the uncertainty of entering into that iteration. And I think one of the key things that you and your team did really well was you do the work early to reduce the uncertainty and then you stop. The natural reaction is people go and do the work, right? They create the data model and that becomes the design that has to be implemented. Where your team are always, well, when I was there and saw it, they were always using it to say, okay, what, in this period, and they timeboxed it, in this period of time, how much early work could we do that reduces the uncertainty and has value? But it's still a guess, right? It's going to be validated when we do the real work, but we just, we have a better insight in what we're walking into. So, I thought that was a really good technique, but you had to get that balance right because otherwise detail people love to go and do the work, right? They love to deep dive and solve that problem.

And to be fair, the way it's working with our external squads is our translators are prototyping, so they get into a huge amount of detail. But actually, that's a pattern that works. You run a small prototype to define the thing that you need to build, and that's what our internal squads do as part of their processes. They will prototype it. Not only does it help you understand if the thing you're going to build works, but it also helps you define your test scripts and what you need to test. So, that's been really helpful having that prototyping mentality. If a design comes through now with this big program that hasn't had a prototype, that's a risk indicator for us. So, you know, we haven't codified like a risk mapping or anything like that, but you know, "Oh, that doesn't exist. It's been done by this person who hasn't seen that domain before. Riskier. Let's provide some more support." Yeah, yeah, yeah.

So, I'm not suggesting you do a stupid risk spreadsheet that everybody ignores, right? But just been able to look for the patterns of the way everybody's thinking about these things. And again, that prototyping process, right? There was, you know, kind of saw a standard behavior where the prototypes never went to production, which is another anti-pattern everybody else does, right? As they prototype it and it's close enough, then they push it and then deal with the technical debt later. With your teams, your squads, they were prototyping to learn more, and then, you know, they rebuilt from scratch pretty much using that.

Yeah, well, actually, that's a cool iteration. So since we've moved our tooling to a much more cohesive toolset that engineers and analysts are working on the same toolset now. The prototyping will typically happen by an analyst because it's closer to their end of the T-skill spectrum, but it could happen by anyone in the squad. Sometimes you'll get a prototype that's so good that the engineer will take it, QA it, tune it, and be like, I can't really, you know, and then, and then that will become part of the production code, which is super cool because, oh, it's just that beautiful mashup of people across that T-skill spectrum working really well together.

But to be clear on that pattern, it's not, here's a prototype, push it. It's the engineer saying, actually, it meets all our coding standards, it means our definition of ready, meets our definition of done, it complies with all our standards, which, you know, most people will pick up those things in the beginning because they make their lives easier. One experiment you might want to try, and I've never been lucky enough to be able to work with a customer that we could try this, is there was an idea of an Agile dojo. I think it came out of Walmart. And so, what they used to do was they had a separate space, or I think it was a building, and whenever they brought on new squads, they effectively went into the dojo. And they actually did their work, but in a dojo environment. So, they were surrounded by more coaches than normal. They were often taught, they mentored, right? It's this whole idea of they were put in a in a dojo space to start the work, but it was, you know, it was different to working and what they go to when they go back into the normal space. I thought that was a nice idea.

I'm smiling because we kind of are experimenting with it at the moment. So, our next iteration has been after this learning from onboarding a whole new squad, which we're never ever doing again, is that we will always start a new squad with someone who's been in the practice. So, you've immediately got domain knowledge, you've got knowledge of how we work, there. And it's more than just the knowledge of how we work. It's that connection. So someone who knows that so-and-so likes mountain biking and go and have a chat to them if you wanna go for a ride on the weekend. You know, like stuff like that. The less kind of technical stuff but the more social stuff. So that's one of the things that we're doing now. We actually did a self-selection exercise with the whole team which I spoke to a few of the guys about afterwards and they were like, "Yeah, I didn't get my first choice or my second choice." I was like, "Oh, shit. I was hoping everyone at least got their first and second choice." Because you have to balance that thing of the right skills, the right the right people fit amongst the teams. But one of the patterns we're trying on this Agile dojo concept, and it actually was our delivery practices lead came on in the midst of all of this chaotic stuff happening. And thankfully, she's been so focused on empirical evidence and listening to feedback that a lot of what we're doing now has come, has been born from us having conversations with her and her coaching us. So, one of the things we're trying now is when we are experiencing, because we're going through another scaling spike at the moment, As we're experiencing that, we're taking people out of the platform squad. So, we're taking guys who understand absolutely how the things work. We're buddying them up or embedding them inside the data product squads with the new people. So, the new people then work on either a piece of tech debt we've got or a ticket that's in the sprint for the squad that they're a part of with— so, pair programming. So, the idea is, I don't know if this is going to work right, we're testing this right now, but one set of hands on the keyboards, two sets of eyes, and the other set of eyes is your platform trainer. That then allows these people to have basically really tight mentoring and coaching as they're onboarding, but they're in the squad. So, the platform trainer behaves as if they're part of that squad. And that's where you start to build connections with the other people in your team. And we're moving away from a 4-person squad, we're trying out a super squad, It's still not that big, it's 8 people. And we might split them up, depends on if 8 works or not. We've tried with 6 and 8 before, and it hasn't, and the feedback we had from the team is just too many streams to think about. We, you know, we want to keep ourselves small so we can be focused together. They like to, quite a lot of the squads like to mob things together. So, this platform training thing is really interesting because the feedback we've had so far from the squads themselves has been, it's awesome because we don't have to carry that burden of coaching someone through all the new stuff. The platform trainer does that. Then from the platform team's perspective, they're like, "This is awesome," because we're in there with them day to day, seeing what's good and not good on the platform. So they're using it as a feedback loop for that whole vision of how do we make the developer experience awesome. So it's actually quite— so far, I mean, we're only like 2.5 weeks into it, and we haven't even done the big, big scaling. This is the first little bit. It's been working reasonably well, and I think that's a pattern that we're gonna, if it works, we'll look at the feedback from this round, but so far so good. We'll try and adopt more fulsomely. So, as we onboard squads in other lines of business, people who are not as, not part of our team, like, you know, in an HR concept, look at how we can adopt that same practice. So, get one of the more experienced team members from platform, to jump in with the new squad, work on something with them to help them learn, help them connect back to the right people in the practice. Um, yeah, that's been really cool because you can see the excitement with everyone. Like, the guys who are in the squads with new people coming in, they're really grateful. I saw one of them today and he was like, yeah, this has been awesome because we can still focus and know that these people are getting onboarded, but we get to connect. And then the platform guys, they're getting to do something different, you know. They're not in the hardcore tech, they're actually helping to build a data product, but in a really kind of collaborative way. Because most of them are innately coaches and mentors. Most people are good coaches and mentors, you just got to give them the chance. Yeah.

Yeah, and that's kind of interesting because that means they are actually a platform and practice squad. So, if I liken it to what we're doing with agile data, if we treated the agile data stuff as just a platform, just a product, we'd never worry about ways of working, right? We'd never worry about how people actually use them, what they do outside and inside, and how it flows. We just worry about the features in the platform. But by focusing on the practice as much, then, you know, we use it, right? We see how people use it to do the job, and then we go, okay, well, there's an opportunity over there to remove some complexity or streamline or remove a bottleneck. But if you're not embedded in the way of working, you just end up building a product that's disconnected. So yeah, for me, it sounds like your platform team are more platform and practice I mean, there's actually like an add to that.

It's not just the way I think about our practice. It's not just, it's more about the experience of using data, I suppose. So, whether you're a developer or an end user. So, one of the things we learned through this other big program of work was actually having a change engagement lead working with us to help us almost act as that feedback point. Because when you're in data, you've got analysts everywhere, you've got people in different teams who aren't even analysts. You don't even know they exist. They're like, I'm super keen on this stuff. So, having someone who can almost create that not front door, but like front door in the community. So we've actually got a Teams site or a Teams, a set of Teams channels. That's the beginnings of a community. Some of it's, it's, and it's kind of grown organically. We haven't been super intentional about it. There've been certain things we've been quite intentional about, but that's the other aspect. So we think about developer experience as one thing that the platform squad has as, as an objective or an outcome. They want to improve the developer experience. The other part of it is as a support crew, We want to make sure that anyone who needs to get data can get it quickly, easily, understand it, and actually use it to value. So, how do you create that experience for people that they don't have to go and find the analyst who knows the answer, who does the thing? And it's not just implement a data catalog. That's not it.

I was going to say, right?

No, it's not. So, yeah, but then how—

What about just a self-service tool? Can't I just drop one of those really cool, sexy self-service tools? Tools in there and people would just serve themselves.

No, but that's like, that's like the old school, like when I was an auditor, that didn't they talk about you just install a platform, but you don't implement, like there's this whole thing where you got to provide that like wraparound service, right? So that wraparound experience.

It's almost talking about data experience. I like that definition of developer experience versus data experience. I think that's a key differentiator. Many years ago, for a very short amount of time, I worked at Xero. And one of the patterns they had right at the beginning, which I thought was really cool, was 'Cause it was in the old days where you had like servers in somebody else's data center, but they were still servers. And the way they dealt with it was they had a threshold of minimum number of users, maximum number of users. And what they did was every time they got to a certain maximum number of users, they added a server, right? And the reason they did that was they were forward provisioning, right? So they effectively bought, the firepower went up, the query time went down for all the users. So it got to the best it would get, it was ever going to be. And then as they added more and more users, it degraded. And it was automatic, right? Well, there were people involved, but there was no discussion about expending on those assets. It was just the way they worked. If we took that pattern, do you think the idea where we proved ANEBA splitting worked, right? We proved that splitting squads in half and bringing people in was far more effective than dropping in new squads. Yes, you lost some stuff, but the greater good came out eventually. Do you think that actually a pattern of wherever you're starting to get to full velocity, you automatically split, right? And then you're over-investing, but when you have to scale, you already have, right? And then you constrain your scale to that pattern, right? There is no stupid idea of we're going to bring 100 people on now. It's like, no, the neighbors haven't split enough. So we can full project that at this point in time, we will have enough stable squads who are experienced and safely able to do this work and enjoy it to do that work. But until then, bugger off, right? Just get in the queue because adding 100 people, no matter where we get them and how good they are, doesn't work. What do you think? Do you reckon the idea of forward provisioning and just constant ANIBA splitting would work? If you can get permission to do it.

Well, yeah, but I mean, I don't know. I am, I'm inspired greatly by nature and natural systems. And so one of the concepts in nature is that nothing will ever grow infinitely. And I know that you're not necessarily proposing that because you hit a natural ecosystem limit where your ecosystem doesn't support it. So if I think about our practice and the way we work is we're not going to need an infinite number of data squads or data product squads. It'll always be constrained by actually what the business needs to deliver, what they need to do, what they're thinking about. About how much money they can afford to spend on it. So yes, I agree, this concept of full provisioning and having an idea of what might be needed in the future and then growing towards that as you need to. But you also need to be conscious of, like, we're probably going to go through a recession. What does that mean? Like, does that mean actually that we're going to get different constraints put on us around hiring practices, for example? Maybe. But you kind of then need to— you don't want to put yourself in a position that you've brought all these people on, it's a great team culture, and suddenly you've got to make a whole lot of people redundant. You know, like, you've got to I think you've got to balance carefully people who are permanently with you and augment a little bit, like bubble up a little bit with great partners and then shrink back down as you need to. Because yes, I think at the moment I see lots of demand and I see lots of unserviced demand and I see so much opportunity, but I am very conscious of the fact that no system will ever grow infinitely. There is a natural limit and there's a constraint in it somewhere. So, you have to respect the constraint. So, yeah, but I think that, and I think the other point is that I never like to do things too far ahead of time, mostly because it's not like one of those build it and they will come situations, right? I don't want to, because, yeah, that's, they don't come. I'd rather be in a, it's not reactive, but it's It's some people could see it as a reactive stance, but it's that feedback loop of, yeah, that's just how far forward you accept your feedback from.

Yeah, I think it's a balance, but, you know, often when I start with a new team or a new organization, I talk about the adoption curve because empirically, I think now I can, I can say I've got enough data where we start off small and nobody really wants to talk to us because nobody believes we can deliver anything. That was our period. Yeah, and then over time, we start getting some success and then the kimono opens, right? The massive amount of latent demand gets flooded. And then, I'm not sure we got to the stage where empirically I can prove that we then get to a balance. I think we do. I think the natural constraint comes in there. You're not going to end up with more data people in the organization than the rest of the organization. So, we'll go from there. But coming back to that co-op, Rob.

Oh, yeah, sorry. But just outside of that also, I think one of the models you can adopt is you Part of it's a risk issue, right? So if you've got so much demand and someone says, "My thing must be done now," you're like, "Sweet. I'll support you to bring on your own group of people to do that." So that's the other pattern. So I listened to one of your podcasts a while ago with a dude called Jurgen Appelo, and he does that unfixed.work. And I really like that as a pattern library to think about how you scale things and how you grow things differently. And so we use that when we're thinking about solving our scaling problem to say, similar to your point of, at this point, you automatically factor in another server. So we've said, when you bring in another squad, because this is about supporting frameworks and coaching frameworks, you automatically need to bring in another people lead or another coach. But really, I want to try and avoid that by scaling in such a way that we don't do that. We become, how do I explain this, the experienced people, the people who help support others to grow those teams within their areas. Because I'm strongly of the view that the best work happens closest to the problems and closest to the business, not in some center of excellence or some silo, the ivory tower or whatever it is. Like, that's not the approach, I don't think.

Yeah, we had one of the podcasts we did was with a Scrum Master and one of his, and he's a professional Scrum Master. And one of the things he said is, everybody tells me I should become a coach. And he goes, I'm a Scrum Master. I love being a Scrum Master. I am the most awesome Scrum Master in the world and I just want to be a Scrum Master. Master. And I think that's the same within teams, you know. So I have a framework where I talk about novices, practitioners, experts, and coaches. And I'm very clear that lots of people just want to stay being an expert. It's very valuable. It's a valuable set of skills and roles. It's a valuable career. And we shouldn't always think that everybody wants to be a coach, right? And so, yeah, so having experts within a team, but allowing people in those squads that want to go to a coaching level and stay in the squad, They don't want to be a coach across multiple squads. They just like to lead and mentor and coach people in their team.

We've got that exact practice happening. Like in one of the squads, you've got this guy, he's just, you give him an SQL problem, he is expert. He is so expert. People go to him like, I've got a problem with this, can you help me fix it? He knows his stuff, but he doesn't want to, you know, he doesn't want to go and become a coach for the whole practice. That's just not his bag. He loves his work. He's awesome at it. He's awesome when people ask him questions, he'll coach in that context. When you compare that with some of the other guys who are like, actually, I do want to influence others and I do want to do things differently. So I absolutely see that happening, that you get people who sit at that level and they love it so much and they're so good at it. Yeah.

So one of the other things that's come really clearly on the No Nonsense Podcast is this idea of setting a clear goal for teams and letting them get on with it. And so one of the challenges that I see regularly when you bring external parties and co-ops or partners or vendors or whatever you want to call them, when you drop a bunch of people in that aren't going to be there forever, who work for an organization that's not the same organization as yours, so that organization has a set of goals that are outside your set of goals, that brings in a really interesting tension. So, how did that work for you? How did— so, I mean, obviously dropping in people that had never worked in your organization before, you had the whole subject matter expert, they were professionals, but they worked a different way, so they didn't know your way of working, is that whole change, which is horrendous to begin with, but how did you deal with the fact of the goal of that organization probably isn't the same goal as your organization? How did you deal with that problem?

Honestly, I haven't perceived that as a problem, and I don't know if it's just because we have such a great working relationship with our partners. I think when we set up this— we call it a co-op for want of a better word, but it's like a partnership. When we set it up, we approached it in a very different way. We didn't do— we don't do RFIs every time we need a piece of work, we, like, it's akin to a panel, I suppose, like a vendor panel. But we've had situations and we haven't fully leveraged this. We haven't fully utilized this way of working yet, but we'll have situations where we'll have a problem. We'll take it to the group and say, hey, we've got this problem we need solved. Can you help us solve it? And typically we'll have quarterly sessions. We'll go in and say, this is what we're seeing coming up. This is what we're planning to do. Here's where we might need some help. Can you guys have a think about it? They will go away separately and meet together as a group of 3 separate vendors. So, and they compete with each other in other respects, but with us, they're a group who work together. So, I appreciate, like, it's just amazing. I'm actually constantly surprised by how this works. They will work together and they'll come back to us with a cohesive approach to a problem and they'll say, this is how we think we need to approach it. So-and-so can't help in this space. Place, but so-and-so can, so we're going to commit our expertise in this way, and this is how we think we can deliver it. Because I think one of the things we defined was we didn't go out and pick a vendor who could do everything. And when people came to us as part of the selection process, we were really clear that we don't believe any single organization can do everything perfectly. There's always going to be some who've got greater depth in different areas than others. And so, we looked for people who were really clear on their strength areas and where they could absolutely contribute value versus those who said, said, "Oh, we can do everything under the sun," kind of thing. And yeah, some of those organizations could probably do everything at great depth, but we kind of had specific problems we needed to solve, and we still have similar problems. So, we went down that route. So, we kind of have almost domains, like technical domains that each of our partners operate in. They blur at the edges often, and they'll talk to each other when a blur happens, and one might be better fit than the other. But yeah, I haven't perceived it as a problem that their goals— I mean, I come from a consulting background. I understand their constraints. And so, I respect that and I don't tell them that we're going to have work next week and then they line people up and then they don't. Like, that's just disrespectful, you know, like, and I think if you maintain that, that, I don't know, kindness, I don't know, just care for each other and respect for each other, then It just seems to work.

Yeah, I think that, I think that understanding their business model and that they're a commercial organization that has to make money, that's their goal in life, and then understanding how that works and then using that as a strength, right? Saying, okay, well, if we work this way, then we're supporting the way you work, and then we get that symbiosis. So look, it's kind of going back to the beginning, right? I always want to go So, you started off with 2 teams of 4, and the team topology was analysts and what you call BI developers, but they were the engineers, right? So, engineering team and an analyst team, right? That was the team topology. As you started building out to multiple squads, how did you do the domain split, right? How did you decide which squads were going to work on what and when? So, not so much the prioritization, but what was the team topology in terms of— because you talked about the squads were end-to-end to to begin with, right, until you started getting a platform squad. So, it was an end-to-end squad, pick up the work, deliver it, maintain it. Then how did you define what a squad was going to work on or specialize or be subject matter experts on? How did you deal with that domain split?

Yeah, so, once we— it was easy. So, the first split obviously was creating that platform squad. That was the first split. And I think then it became a function of who had existing domain knowledge and who— it was basically playing to people's strengths, really. So, if you think about one squad Actually, to be fair, I'll take that back. Right at the beginning, it was just a matter of let's just keep our heads above water. And thinking right back to it, the very first couple of things we did, one was a customer product matrix, which to be fair was within that domain that that squad was knowledgeable in. The other one was around contact center data. And again, to be fair, the guys in that squad had existing knowledge of the— So, yes, we did. We did play to the strengths of people's existing knowledge existing domain. And one of the patterns, one of the things, one of the reasons why we've done— oh, there's so many reasons why we're changing to the super squad way of working at the moment and this platform trainer way of working is we noticed that the cognitive load in the squads was too big. So, even after we brought the third internal data product squad in, they, because they were all new, we didn't overload them. We gave them one subject area, pretty new, pretty green fields and let them go with that. But it meant that our other two squads ended up with just ever-expanding domain and ever-expanding cognitive load. And it becomes, especially when you start ending up with integration-type data products, it becomes like, you know, there's lots of spikes that turn up and that can be difficult. And with, you know, the COVID situation, I don't think we've had complete squads for in any of our squads for the last 6 months. We've always been running sort of 75%, 50% because people have been sick or away. So, one of the things that we, as part of what we, that was the third thing I couldn't remember before that we needed to change was we just need to reform because people are carrying too much. We need to split the domains a little bit more because it's too much. And yes, everyone crosses over, they touch each other's domains, but that's where having that data design forum helps. You keep that that consistency of your source of truth. So, that's one of the things that we're actively working on is which pieces leave which squads and who adopts them. So, one example is that new squad was working on kind of more customer touchpoints. They'll pick up a whole of the contact center type data as they'll become their domain. And that will reduce cognitive load in the squad that originally developed it. Because it's now nowhere near the domain that they operate mostly in. So, it's about who operates mostly in something, who has the most expertise, but they're not the only squad that can do it. They can all do all of it. It's just a matter of quality, speed, and time, right? So, anyone, it's not the best cab necessarily, but it's the next available cab off the rank might do the work. So, yeah, so I think the way we think about it is place people's strengths, find, and what do people enjoy doing? Because that was part of the self-selection exercise was we told everyone about what the work streams are working on because our squads are now embedded within bigger teams of teams. So, it's part of a change that's happening within our business. And so, I gave them an overview. We had the product owners come in and talk to us, and this is the work stream kind of product owners come and talk to us about what the streams were working on, what they were looking at doing. And then when we did the self-selection exercise, we said to the guys, this is what's happening. These are going to be the kind of of objectives or outcomes that those streams are working towards. Here's roughly what might be on the roadmap, but, you know, it's subject to change because this is the first cut. Find something that you want to be engaged with. And it was really useful because some of the guys in the team were actually disengaged from the things they were working with. They're like, ah, I'm a bit bored with that now. I want to work on something different. So, it was quite a cool opportunity for people not only to play to their strengths, but to find something they're actually interested in. And yes, it didn't work out because some people didn't get their first choice, but you know, you can try.

Yeah, but, but yeah, there's always a risk that once you get patterns that are working, they become immutable, right? So you just double down on them and you never sit back and go, okay, let's take a fresh look at this. Let's rub out some lines and redraw them because, you know, some change is good, right? And, you know, change is as good as holiday sometimes. So that ability to sit back and observe and go, okay, We're not just going to double down the practices and patterns we have right now as we scale, but we're going to look at them and say, what would we need to change?

Yeah, well, I think that thing of you have to listen to the feedback, and the feedback we're getting from the teams is very much that. And it wasn't a stagnation, but it was a bit of a, uh, this again, you know, like, and so that's not great. You don't come to work to say this again. You come to work to do something cool that you're engaged in and that you really feel compelled with. Yeah, so you listen to that and then you take action to help resolve if that feeling exists.

So, watch out for the been there, seen that, got the t-shirt. All right, just to close it out, go back to the community, right? So, we talked about, you know, the squads and the co-op and that, and then your community, this ability to have people outside the core data professional group engaging the literacy and that. So, tell me how you've gone with that one. Cause that's always a big— well, they're all big challenges, but that's yet another big challenge.

Yeah. We like good problems, really. So, I think it started off really as a change management exercise, and I hate those words, but when you've got a problem, like you're changing some tools on people, you're changing a way of working, you want something different to happen, typically, if you're in a project management world, it's change. So, how do you manage the change? So, we flipped that around a bit and said, this is a cool opportunity. Where we're changing something so we can change how we work with that something, so we can implement a new paradigm around it. So, rather than just putting a self-service tool in, let's actually put a self-service tool in and create a community around it. So, one of the examples was when we implemented our analyst tool, our analytics tool, we did this in cohorts. So, we started with a couple of groups that were really close to our team, and we'd worked with them really closely before. We knew they had had good base skills and we could easily move them into the new tool. We ran intensive kind of 1-week training sessions, which we got support from our partners with. It was sort of morning time, sort of 2 hours, come in, we'll do in-person training. It was all based around some kind of online learning that they could self-pace if they chose to, but it was almost like a cook-along kind of thing. We did that, that kind of intensive 1 week, and then we had weekly drop-in and we still run those. So, that particular community is really strong. We've got a Teams channel with it. If people have a problem, they'll drop in and say, "Hey, I can't get date formats." I'm an accountant, so in Excel, I know date formats suck, but apparently, date formats kill everybody. So, you know, people put comments in like, "I can't get this date format to work. Can someone give me, like, help me out?" And then, it was so awesome. We saw this chat that went to 27 different interactions, and it went from an analyst asking a question about out, how do I deal with this problem? Then another analyst popped in, not even part of our team, part of a different team, who'd been one of these cohorts, popped in and said, oh, have you tried XYZ? Then an engineer popped in, was like, there's actually a much better way to do that, try this. Then the first analyst popped in and said, oh cool, I'll try both. And then all of a sudden, they're like 5 different people, and people had learned from this interaction. So I think it was, we had to, what's the right word, you have to nudge it. So the way we nudged it was by having regular sessions that encourage people to come. So there's got to be a what's in it for me. Like, no one's going to give you an hour of their week to come to a drop-in session if they don't see value in it. And so what we found is there's ebb and flow a little bit. Like, if there's a bit more busyness in the business, we'll see that lots of these, um, business-based teams won't necessarily turn up. And we'll say, ah, not much to talk about this week, let's not do it. Um, but there's always a topic, there's always a share. So So, our lead analyst will line up someone to talk about a problem they've solved or a new insight they've gained. He'll line up that to come and then talk about it and share it. And that's worked really well. We also have a little bit more of a, not exclusive, but a slightly smaller community with advanced analytics where we're trying to push analysts who are excellent in their domains, who've got a desire to learn more and grow more, to develop more statistical skills. So, push more into that data science. ML type space. I mean, so that's a smaller community where they share deeper insights, more kind of grunty learnings than the one that's run with the drop-in clinics. That's open. And that's the other thing, openness is important. Like, this is not like— this is not exclusive. Like, these drop-in clinics open to anybody across our business. And what we found was our community grew much faster than we expected it to because our focus groups was one area of the business. And then all of a sudden, the other side of the business just started turning up. We're like, "Oh, hey, nice to have you here. Let's talk about some of your problems." So, you know, like it's been really cool because you get different perspectives. So, we've done that for the analytics tool, the Explorer tool. We've seen a little bit of it pop up around the engineering practice as well. And as that grows out, as we get more product squads embedded within other parts of the business that don't, that aren't in our core group, that will become more and more important as you start to not standardized practice, but at least have some consistency so people can talk to each other effectively. Yes, I think that's been important. We do kind of, we do quite regular posts about sharing success stories. So we do a bit of storytelling. So if someone's had a win, we'll actually make it a little bit, it's a little rah-rah because that's just how we are, a little rah-rah about it and share that. And it's kind of nice for people to be recognized when they do some cool stuff. 'cause often they just do it, move on to the next thing. And there's some amazing stuff that these guys are doing and you just need to recognize it. So we do that. And then one of the other things that we did was we actually did a full-on community building event. And it didn't quite work out 'cause it was when we experienced a little bit of a COVID spike. But we got drinks and pizzas, like beers and pizzas. And there was a task at hand. The task was, 'cause part of what I wanna try and do is with this community is identify not product owners, but people within the business who care about the value that data can deliver, and then get them to help us articulate that value well. So, for everything that we've got running on any of our platforms, on any of the tools, I want to understand why does it exist and what value is it enabling. Because if it doesn't exist for a good reason and it's not enabling value, well, why Hey, does it exist? Um, so what we did is we called it Data Insights, um, Data and Insights Sharing or Inventory Creation, some boring-ish name. But the sell was come meet others in the community. We paid for people to come from down south, up north, to come together. And yeah, bad weather, COVID meant that not everyone was there, so we ran full hybrid. Everything we do, we run full hybrid because we want to be open and accessible. And we started off, we said, here's a simple SharePoint list with some simple fields. Fill in for everything you can think of that you've been using the new tools or the new community things on, everything you've done. Because, you know, you do so much in your day-to-day that you don't even recognize is valuable, but it is valuable. It's created something. So, we did that, had some drinks, had some pizzas, And it was amazing because that created the sense of belonging. And we've had people more and more in the customer research space go, "This isn't just a data and insights community. It's a data research and insights community." I'm like, "You're right." And they've been talking about it as like a "we" thing. And that makes me so happy because it's not a "me" thing or "my team" thing. It's a "we" thing that— because the ultimate goal, like I said before, is this needs to self-sustain. We should all be able to step back from this and people are so connected and they're so passionate about the things they're doing, that they can get it to self-sustain. So, yeah, so the next iteration for us on that is figuring out exactly what needs to be true for this to self-sustain. And you can see pockets of it where there's more maturity in it. We don't even— we almost don't even need to do anything. Then pockets of it where there's less maturity and you kind of do need someone in there. But you do, you need, you need your advocates. You need those people in there who are going to give a little nudge every now and again if stuff's not moving, or if you see things quieten down, to understand, well, why is that quietened down? Is there a product Is there a problem with the product or service, like the data experience? Like, is there something wrong with it that we need to fix because it's quietened down? Or actually, are they just busy and they haven't had a chance to connect? Maybe we just need to set something up to connect them. So, you know, like, I think, yeah, it's been faster than we expected. Like, many more people came into it than we thought we would get. And I think that's mostly because we've just been open about it and we haven't been exclusive about it. And we haven't been wedded to tools. Like, yes, there was a change we were making to bring some new stuff in, but at the same time, we've got parts of the business who are not working on these platforms, but we're embracing them as part of the community because they know stuff that we don't. And that, yeah, so it's super cool. I love building. I love building communities. I just love this stuff.

Yeah, I think, you know, we could go on for hours and we probably need another one where we talk about community building cool stuff you did. But one of the things I pick up on there, and it's a term that I've stolen from you and used time and time again, um, and your love of nature, is that idea of a permaculture. That actually one of the signs of success, one of the things we should strive for, is when the people who started have left and it keeps on building without us. Uh, and that's what we're striving for, right? Is that we know we're successful when that permaculture is there and keeps building and iterating and changing the way it works without us. So, So, that's pretty cool.

To add to that one is, if you understand how trees are connected networks, right? So, a tree by itself stands, but underground, it's connected through a whole network of microbiology. I'm not going to use the right terminology, but it's effectively like a fungal, a fungus network, and sounds a bit gross, but it's super cool. So, if you think about it, what we're trying to get growing is that network that exists and connects all of the things you see above the surface. So, you know, to the leadership team, they see these cool things that create value, but below the surface, There's this connected network of people. And if you think about nature as a system, network systems always have, they're the best way to get resources to flow to the right place and get knowledge to flow to the right place. And so, if you can create a networked way of working, and I am not going to use the word data mesh because it is not that, it is a networked way of working, but maybe it is, and maybe I don't understand that concept well enough. That's the idea, is you create that network that sits below the surface, that's semi-visible, visible, but actually that's what the whole thing thrives on because without it, those trees are way less stable. They're way less effective. You know, you can see a forest that's got a great underground system versus a forest that doesn't. Like, it's just amazing, the difference.

Yeah, look, I think, um, you know, building something like that where, you know, the network is supporting it is great. We've got to be careful we don't end up being ducks, right? Where we're paddling furiously under the covers and we look like we're doing And, you know, you could take the journey you've done over the last 4 or 5 years, you could do a presentation at a global conference how you've adopted data mesh, but realistically, what you've done is figured out principles and patterns that enable you to distribute, decentralize the work at the right time when it makes sense and with a whole lot of testing, experimentation, and making sure it's safe. So, yes, you could go down and pretend you've done a data mesh, But, you know, I don't believe you're the type of person that will pick up the latest vendor bullshit and try and pretend that's what you're doing. But the principles, right? The principles of self-organizing team are in control of what they do, the principles of splitting up based on domains, the principles of platform as a service, and the principle of— and we didn't even get on to federated data governance, right? And no stupid committees, which which I think we'll get you back for another podcast on that. If I look at those principles, those are the principles you've been aligning with and trying to implement for years. But, you know, let's not just go and pretend that we've done data mesh, because, you know, you've probably got more patterns for how to do a data mesh than anybody else I know. But it's just the way you work, right? You're just striving to achieve those core principles because those principles have value.

Yeah, and I suppose, like, I think, if I think about our team, I don't think any of us would say we're anywhere near there yet. And in fact, I would even say we're not anywhere near there yet because we can always see ways to make it better. We can always see ways to be more effective. And a lot of the things that we're doing, I suppose this is part of one of my failings, is like I tend to, oh yeah, that's awesome, that was great, recognize the success quickly and then move on to the next big problem. But, and I think that's just a pattern. Like we want to shoot for the stars and we want to do some awesome stuff. And so yeah, when people say you've done some good stuff, like, oh no, there's so much more we need to do. Like, it's one of those things that— but we have, like, I think if you look at, look back at something, some of the guys say if you look back to what we're doing 6 months ago, it's a world of difference today. And I don't get to see all of that anymore because I'm not, you know, it's not 8 of us anymore. And so you get little snippets of it from talking to people, and you see how things have changed, and you see how engaged people are in it. And I think for me, it's that thing of of if you can get your network system of people working really well together through all those things like coaching, team coaching, systems thinking, all of that stuff, then it's just, I don't know, you never will be there because then stuff changes too fast and there's not a there. So yeah, I don't think we're doing more than anyone else. I think we might be converging on similar things to other people because it's like nature, right? You get a dolphin and a shark, they both swim beautifully fast in the sea, but they've converged on that through completely different evolution. And it's the same as our team. Our team's converged on this way of working, as other teams I imagine have, and maybe they don't talk about it either. So, yeah, thank you.

Maybe the teams that are doing well aren't the ones talking about it, right? They're the ones doing it. Maybe that's the key pattern across the world.

Maybe, maybe, maybe we should never do another podcast.

No, I think we should share, right? I think we should share, and I'm going to hassle you to do more because I want to help everybody understand the patterns that you have applied. Not the principles, because everybody knows the principles, but the patterns, the things that you've experimented with that had value in a certain context, and the ones that you tried that didn't have value but may do for somebody else, right? Because that, for me, that's the gold, right? That's what helps people, is describing a pattern that they can experiment with. So look, been great, hour and a half. Thank you for your time. As always, I knew we were going to go over time. That's right, they're good rabbit holes, right? And we do come back out. So look, thank, thank you for coming on finally. I really enjoyed that, and we'll catch you all later.

Yeah, thanks, Shane.

And that, data magicians, was another Agile Data Podcast. If you'd like to learn more on applying an agile way of working to your data and analytics, head over to agiledata.io.