AgileData #2 - Adopting the AgileBI way with Sam Sewell

Welcome to the Agile Data Podcast, where we talk about the merging of agile and data ways of working in a Simply Magical Way. Excellent. Well, today we've got Sam Sewell who's come to join us to have a little chat about the thing we love to call the Agile BI Way. Sam, welcome to the Agile BI Way podcast.

Hi guys, glad to be here.

Hi, Sam.

So I think probably to start off, maybe you could just tell us a little bit about your background and what you've been up to for the last 10 or 20 years.

Absolutely. So I've been mainly involved in the data BI reporting world for about 20 years. It was my first actual office job was data entry. And from there I moved into, can I get a report on the data that I've entered? And somebody said, hey, that's kind of BI. You're now a BI person. And I kind of I've kind of grown in that way. So I didn't study this at uni or anything. So for around, yeah, 10, 20 years I've been involved in this, and I don't think I knew I had a profession for probably the first 10 years. It probably wasn't until I met yourself, Shane, back a good long time ago, where I started realizing how much depth and how much knowledge and history and how much of a big industry and profession this was. And in the period I've been working in this space, I think I've touched on a number of different areas of these disciplines. So I've had roles looking after platforms, I've been a consumer, I've been someone who writes reports, I've worked with people doing modeling but probably not smart enough to do it myself, and I've managed teams across a lot of different technologies. So like, I do love what I do, and I'm always interested in where it's going.

Oh, that sounds great. It sounds very similar to my background actually, Sam, and banking and dealing with data more and more in every role that I moved into until I kind of realized that I was a data person. Yeah. So when did you actually hear about this thing called agile BI, and when was your What was your first experience with it?

Well, Agile BI has been really recent for me. I mean, I only started hearing about Agile about 5 years ago. So I've worked in banks, I've worked in some public sector, I've worked in energy sector, and I think that probably some of the leaders for Agile, in my understanding, in Wellington where we are, were some of the tech companies who were embracing— they were small enough and fluid enough to have a good go at making these new ways of working work and be effective. So my introduction to just Agile itself has been very, very slow and gradual. And I'll be honest with you, I know most about it because my partner worked at one of these tech firms and she embraced it and she learned about it and she blogged about it and she really introduced me to what an effective way of working it can be. It's really being mindful about the way you work was one of the big benefits of agile. It's working more deliberately. And then agile BI was something I started hearing about with some of the people we've got in the market here, some of the luminaries in this country who were starting to think about applying agile and BI together. I encountered it myself a few years ago when I was put in a position to be working in an agile BI team. But none of the people had necessarily been agile before or had thought about how you apply BI to agile. There was a lot of head scratching and a lot of challenges we knew we had to work through in that space, but we didn't have the answers. But luckily there are some people on the market who are starting to come through with ways of meshing BI and agile together to make it work.

So when you were looking at that, you know, back in the early days, I mean, How much content could you find about how to actually apply Agile in a data space?

Well, next to nothing.

Right.

Next to nothing. So it really, again, it's been the conversations with you where I've really come across someone who's put some proper thought into it and then has had an opportunity to work with it in the market in real-life situations. Because there's lots of things on paper, on books, there's lots of people thinking about things, but you really need to connect with people who have done it in anger on programs, on projects, in real-life situations.

So given that we've just been through a project where we've kind of, or in the delivery where we were definitely doing it with anger, tell us about your experience with some of the patterns that we kind of worked on together and started implementing with the team there.

Yeah, so overall it goes back to me to two things. One, it's taking a new approach. Because I think for most of the time I've been in BI, we've known there's challenges about how BI programs and projects run. We know there are delivery challenges, we know there are scope challenges, we know these are some of, some of the more difficult projects that you'll work on in organizations. So having a new approach, having a new way to look at old problems was really exciting. It gave you a chance, okay, well maybe this is the one. Because I've spent 20 years working on BI projects where we've not always been as successful as we'd like to be, but maybe this is the one, this is a paradigm shift, a complete different way of thinking and looking at the problem. So that's been really exciting, and then because you're doing that, we go back to working deliberately. You're not just, here's some problems, let's try and solve them. You're looking at approaches, you're looking at techniques, you're looking at tactics, to approach the work to help it be successful. So the work we did recently, I think the way that the approaches, the patterns, the techniques were able to connect with the teams who were going to be applying them, resonate with them, give them hope that you've got a good chance, because you don't want to start a program unless you really believe you've got a good chance of putting it through. So to be able to put that into people, to be able to bring them on the journey, to say we're going to do this and we're going to do this and this is how we're going to do it and this This is why we're doing those techniques, I found was a really good way of doing it. The approaches, the patterns, the techniques that we were implementing all seemed really solid. They all had good background behind them. They were all there to fix specific challenges that we've all seen before in the past.

So the business, so the product owners that you brought along on the journey with you, What was their sort of reaction? Were they experienced with this previously, or was this all new to them? And how did they sort of like come along with you on the journey?

So a lot of this is new to a lot of people, and in programs you'll often have people come on who aren't necessarily BI or data people background. So it worked well because all the tendrils start coming together. This is what we're doing, this is 'Why we're doing it.' 'This is how we're going to do it.' So we found with the techniques it was, it was relatively straightforward to bring people on the journey for how we're going to do it, why we're going to do it, because often people are looking for someone with an approach, with a vision, and this is all part of a vision and an approach and a technique. It's a— it doesn't have many or any sort of gaps in what you're doing. You can see a path, you can see the way that you run the workshops, you can see the way that you elicit and capture the information, why you capture that information. A requirements document, an old waterfall requirements document, it's got lots in it. I don't know if there's— it's always simple to justify why you're putting all those bits of information, why you're gathering that, putting it in a big document. It's not all going to be needed, but this approach was everything had its purpose. I need to ask you this question so that I can make this decision and do this piece of design.

Sounds great. Yeah. Yep.

So with the team you were working with, you started tailoring the patterns or the processes. So, you know, you kind of started forming different ways of working by using those techniques in ways that I hadn't seen before.

Do you want to tell us about some of the things that you saw the team kind of do So yeah, we found that we would take the ways of working and we had to allow the incumbent team members, the people who worked at that organization, an opportunity to take ownership of those ways of working and invest in them so that they became their own way of working. And when you do that, it allows them also to do some tailoring to put it into the vernacular of the organization and help them sell it within the organization. So you take this great way of working and then you put your own organization's branding, ways of thinking, ways of working on top of it. So it's still very much the same underneath. It's still very much the same approach, series of techniques, patterns, and processes, but it's branded for that organization, for that team. And then you allow the people in the team to make small changes and edits, keeping consistent with the with the philosophy of it, but in such a way that enables them to sell it and push it and go out there. And it really has taken off in the organization. The terminology is the labels for the information products and the labels for the type of workshop and why they have them are being adopted at all levels of the organization because it makes sense to everybody and it resonates with them. So Yeah, it was a very simple set of techniques to apply even to a very large organization.

Hmm, yeah, so it's kind of like a best fit for the organization, not this is the Agile Manifesto and we're going to follow this to the letter. And we found that in my organization as well, that if you just tailor it a little bit to get a few people across the line, then it's worth it, definitely.

Yeah, well, the challenge is buy-in. You can have the most amazing approach, and I've seen Agile applied in organizations where it hasn't landed because the people around— change is the hardest thing in the world in organizations. Change the way you're thinking, working, come to an acceptance that the way you do your job today is not necessarily the most effective way of doing it. Not everybody's open to that kind of change. In all organizations, you get people who are career people in the field, but you also get people who are there because it's their day job and they're not necessarily students of what they do for a living. So you've got to be able to excite everybody in the team and have everybody on the team on the journey. But you do need to keep the philosophy. You need to keep the why. Why do we have this way of working? How do we structure our conversations to capture the why the whole time? The same, but you can change sort of the colors and some of the looks and the feels of the thing, but it's the philosophy that's key.

So on that, I mean, often what we find is, especially when we work into organizations where Agile's new, there's a vision, the desire to behave differently, to work in a different way, and teams of people are normally then mandated or seconded or encouraged or dictated to work in this new way. And for some people it's really uncomfortable, but they make that change. And for some other people it's really uncomfortable and they don't make that change. Did you find that? Did you find that given that a team kind of went on this journey, that there were people amongst the team who opted out? And how did you deal with that?

Yeah, so So we found that actually a higher percentage of people were able to pick it up and adopt it than I was expecting. And because it does speak for itself, we're not trying to sell fairy dust and magic. We are— someone says magic doesn't happen here. I can't remember who said that. Someone I know says that on a regular basis. But you're selling something that's that's got good reasoning behind it. So a small number of people really struggle, but that's an organizational challenge, I guess. Often I found the people that aren't as open to picking up new ways of working will find a role for themselves to still be effective in their role. I've never found that I've been able to get 100% sort of change everybody on board, and you don't want to go in and necessarily change the people in the organization, but I do believe there is a pot for every lid, as it were. So the people that are not able to embrace it, I think it's more of a time and an exposure thing, but in our world there's always work that needs to be done. So it's about harnessing the energy of those who are excited by the change and hope that over time they bring everyone on the journey in a show, don't tell way. So there may be skeptics, but you can, if you do things really well, you will bring people on the journey. That's my hope.

Yeah, so we've found that the retrospectives actually expose people's lack of buy-in, and we've used retrospectives to try and change things to fit everyone into the way of working and get everyone feeling comfortable and having those tough conversations as a— after your timebox period is finished and you're looking at improving the next one. So have you used retrospectives in your workplace and how have they gone down?

Yeah, we absolutely have. I think we probably could have run more. And retros, I think, are one of the ceremonies in the Agile world that can be run in very different ways and have very different feeling around them. I know my partner, I think, bought a series of flashcards to run retros that gave you lots of different ways to phrase the conversations and get people to interact. So I think the retros take a little while to get going to be effective, and I'm not sure I've seen them reach that stage yet where I currently am. I think they exist for a very good reason, and it's part of the change to get people to that openness and that honesty to work together, to not to blame, to not to point fingers, but to collaboratively work on better ways of working moving forward.

Yes, I think with retros, it's interesting from my point of view, you know, retros tend to come out of using Scrum and But they seem to also have the most available list of games in the marketplace. In terms of making retros fun, there's lots and lots of places you can go to find awesome games for retros. Not so much for backlog grooming, story planning, or any of those other things that come a little bit more boring. But yeah, flashcards and different retro games is kind of exciting.

Yeah, I enjoy them.

So obviously, well, but as part of what you did is you've you've come along on a journey with that current process or project that you're working through. What would you say is kind of the hardest part that you, or hardest thing you've encountered, kind of introducing agile ways of working into an organization that's going through massive transformation and is probably going on the beginning of their agile journey as a business versus, you know, the difficulty of moving to an Agile delivery method?

So I think if the organization was already there for Agile, if I were to work at one of these tech companies that's been doing Agile for over a decade, everyone in the organization's bought in, I'd say the most difficult thing would be transitioning the team and getting them to embrace the new ways of working. But when you're actually part of the vanguard into Agile ways of working in your organization, you not only have to change the way the team works, the way your stakeholders work, but also really the way the enterprise operates, the way it thinks about funding deadlines and balancing organizational change. So for me, the most difficult thing probably is that wider senior stakeholder management of setting expectations, of finding space to learn and grow grow and move while still being able to consistently and constantly deliver to the organizational needs. I guess that's always been a problem, but it's also fighting the temptation for me to just go, "Oh, okay, fine. I'll just get a whole load of old-school developers and we'll just put them in a room and we'll just— because this is really important, so we'll do this bit waterfall just to get it done." So having to fight myself a lot to sort of hold my line, hold my course. To push the change through.

Yeah, and I mean, I found the same thing as I change, as I move across organizations and I start moving into a new organization to help coach them or their teams. It took me a while to realize that at the beginning I was focused on Agile delivery. I was focused on the team and how the team works and getting the team up to be self-organizing and responsible and self-aware and optimizing themselves and and bringing that change to that team. But actually, there were two things around that that were just as important as agile delivery. There was business agility. So the number of times you work with an organization that talks about a project with a funding for a window, you know, heaven forbid a project manager or program manager and a Gantt chart, that sits above an agile way of working. So it's the ability for that organization to become agile and change the way it operates. Then the second part was kind of the after the agile team has delivered is getting everything into front of the user. So, you know, automating the deployment, automating the operations, stopping the problem of as the team delivered more and more, they were often held responsible for maintaining it, and that wasn't automated. So we saw kind of drain, right? We saw, you know, the team's velocity for new stuff kind of die as the BAU bleed, I call it, happened. So for me, yeah, we focused on agile delivery to begin with because that's the problem we wanted to solve, but now actually it's realizing that business agility and automation of the delivery and ongoing management of it is just as important. I mean, you probably found that as well, Blair, in terms of, you know, going from the early days where the teams were just rocking it to, you know, now you've got stuff to maintain because it's the same team.

Absolutely, yes, and you've got to balance that with the new and shiny stuff, we've found. And the product owners haven't, I've found in our setup, they haven't let go of the product, so they still feel a sense of ownership because they went through the whole thing with you. And some of their refinements we've had to squeeze into building new stuff, and it's just that whole squeeze. But yeah, the thing I've noticed is that ownership aspect, that they don't just go with you on the journey and then just go, boom, thanks for coming, we've built this great thing.

But that's great as well, right? Because now you've got somebody who actually owns the product, right? Somebody's going to fight for adding to it. It's not going to sit on the shelf. It's not like when we used to buy enterprise software, right? You go and buy it, sit on the shelf, you know, you come back a year later and nobody's used it. It's like, wow, that was a value. But then you've got that adoption problem, right, as you get more and more product owners that want more and more done, and you've maxed your velocity. How do you scale, right? What do you prioritize? And actually on that, so one of the things that was interesting for me was the number of information products that you actually defined up front as part of your roadmap. And the fact of, because of the type of work you were working on, you had multiple product owners, each with a relatively heavy list of information products to be delivered. How did you manage the prioritization across that nightmare?

It's difficult. Yeah, it's difficult. So, to your point a few seconds ago, I, to play some— I don't think we've reached that crunch point where you're trying to balance delivery with the support, and I think that's a challenge I know I've got coming up, which I'm excited about tackling. With the IP, so we've gathered a good few hundred IPs and there was a real burst of excitement initially with the IPs because again you're giving the business a good vernacular. One of the ways that I describe these, the BI Way, the agile ways of working, is it's a common language between the three groups that you're working with, between the people in the business who want the outcome, the analysts who are there to facilitate that outcome, and the people in the information or data teams who are the custodians and the preparers of the information. It gives all of those three groups a way of speaking to each other that makes sense to everybody, which I hadn't seen before. You're always talking different languages. This gave it a conform language that everybody understood. So there was huge excitement around the IP. IPs. Everybody wanted an IP, everybody wanted a workshop, everybody wanted a session, and we flood in lots of these. One of the challenges was probably consistency and quality because it was so exciting to go out and run as many workshops as you could. It's how do you ensure ongoing that you're working in a consistent way, which is part of why you have the workshops, is to work consistency— consistently, but you can still have different people interpreting in different ways. So that was one of our challenges. Prioritization, I think, along with a lot of agile ways of working, we have to really road test it to see if it is successful. Transparency and visibility are one of the keys that we've been trying to work on so the whole enterprise can understand what is on the backlog and can understand the import of everything on the backlog. And then it's finding the right group of stakeholders that at the right level in the organization who can collaboratively work together to balance that priority. But that's still, I think, a work in progress for us. We definitely are mindful of it, and we're definitely getting the conversation rolling around the organization and within the governance groups to let people know that they need to be aware that this is working. We have our backlog, we have it in an agile tool, in Jira, So it is very accessible. We keep it up to date, but how that journey is going to go over the next year for how we work through that, I think that's something I'll come back and tell you.

And I think one of the things that you did that I really liked was, yes, you've got your backlog in Jira and you've got boards and it's visible, but you know, in my opinion, Jira is not the easiest product to look at and understand. So one of the things I really liked was you almost took an infographic approach to creating some, some really high-level infographics around the IPs, the backlog, ways or lenses of looking at it to say, you know, maybe by, you know, kind of event or by subject area or by— you put some lenses on there that explained the different ways the backlog could be viewed. How do you find that in terms of, did that effort actually come up with value? Did the stakeholders actually look at it and understand a little bit more how big the elephant was, the different ways that they could prioritize, what was first and what was second?

Yeah, absolutely, and it's something I try and apply whenever I'm working with BI teams is to use BI. We are the BI people, So we should be using our own techniques and tools internally, BI on BI. So creating infographics, being able to pictorially demonstrate if we do A, that unlocks B, which unlocks C, D, and E, and put it in colors, patterns, shapes that are accessible to everybody, that people can put in their hands. As much as we're digital, people still want things in their hands. In their book that they can open and say, well, this is where we are, this is what we're working on. So being able to give people pictorial aids that show the different lenses, as you said, around subject area, around if we— what unlocks other things, because at the start of a warehousing project you have nothing and you can't get right down to your nuanced products until you've done all the foundational work to to show how all of that stacks up and adds up to grow something that's greater than all the component pieces in a way that resonates with people, we found really effective. So I think if we hadn't done that, life would be a lot more difficult for us. It's been a really good sales tool for the team and what they're doing.

So I think also for me, you know, when I do my personal retrospective around the Agile BI Way, one of the patterns that we kind of came up with during that process, and it comes back to your comment around scaling the facilitation of the information products. You know, as people kind of do it the first time and you get more than one or two people kind of running out and trying to scale that facilitation, the quality of what they capture becomes slightly variable. And I think in hindsight, it wasn't until we went into the sizing workshops for each of the IP where we actually sat down together with the team and actually talked through each of the information products, that really the gaps started to become visible, and the people that gathered them realized where the focus should have been because they knew, actually got more of an insight into what the outcome is. So I think, you know, for me, extending that pattern out to, if we're bringing new teams on, go, you know, show them how to do the IP facilitation. Worked through the first couple with them, let them go out on their own, but very quickly bring it back to a sizing workshop. Because that, you know, by talking through it and people who weren't involved in the facilitation actually being able to ask those questions, that I think helped refine the process for the people that went and gathered it, right?

Absolutely.

And made them better. So if you look at it from that point of view, you know, with the next piece of work you're going to do, When you're going to bring the Agile BI way to another team, what do you reckon the one thing you'll probably focus on improving next will be? Where's the gap? Where's the thing you're going to bring to the world as one of the patterns that the rest of the Agile BI-ists, or whatever you're going to call them, can leverage?

That's a good question. And I think—

and not one that we wrote down for you, so you're on the spot.

And it gets us all to think.

Yeah, yeah.

Bribery and threats, I think, is something I'd like to utilize more in the workplace. But no, I think there's, I think very much a show, don't tell very early on, I think, is going to be maybe an approach to take. So as soon as you've got enough arms and legs and heads in a room that you can actually run through these techniques and get people access to a system where they can put them to practice. I think being able to create a really transparent use case where you can say, this is what we did on this day, this is what we got, look at what we can do. Because a lot of the experience in the BI way was selling the value of it, and a lot of people grabbing onto that and really understanding it, but for the wider organization there was it wasn't visible, it wasn't transparent. So being able to actually do an early proof of concept, to go all the way through and prove to the doubters around you and get people on board, I think is what I'd like to do. Because we tried to do that where we are, but I think probably could have pushed to do something faster earlier.

But do you think that's because although the organisation said they wanted to be agile, they really weren't? Business agile. And so, you know, there was an outcome, there was a number of information products or dashboards or reports or datasets or API. There are a whole lot of things that needed to be delivered, and it's just the fact that you and the team decided you want to take an agile BI approach to deliver them, you know, in a better way, that therefore actually a lot of your time was selling to the organization why this approach is actually valuable, why you're investing in these teams doing things slightly differently. Do you think that's kind of why that was the focus area for you next?

Certainly, yeah. I mean, a lot of energy goes into that sales work. So if you could, I mean, you could almost deliver two different ways at once, right, early on, to make it really, really clearly articulate the value of the ways of working. Working. But yeah, that overhead of selling the techniques to the stakeholders was where a lot of the energy goes. If you can get someone to do that work for you so that you and the team can just plow through the work, that would be great.

Yeah, we've found that as well with the concept of the MVP in front of stakeholders early on. Gives them something that they can report back to their to their managers and it's actually becoming a thing. It's quite exciting. And yeah, I think they'll be working on the MVP a lot closer next time too.

So we've talked about MVP/IP. Yeah. And that was one thing I saw at a site I was at previously where they were able to, to get one number, one number all the way through. And they weren't using a true Agile BI way of working. They were doing traditional BI and trying to apply Agile principles to it. But they were able to take one number all the way through and put it on a dashboard. So it was a dashboard of nothing except one number, but they were able to get that number through in one sprint.

Yep, and we always say, you know, vertical slice, thin slice all the way through if you can is is what we should do, right? Sometimes, you know, for 101 reasons, it's not an approach we can take initially, but if you can, thin slice it all the way through. And the other one, I think, picking up on your point, you know, we talk about being ad hoc, we talk about doing agile, and we talk about being agile, but it's a journey. I'd rather hear the words, we weren't really being agile. If we came in and somebody audited us of our behaviors and the way we working, you know, we're not going to come through with a, you know, the retrospective would have lots of things we could do better, but we're starting that change, right? We're starting to adopt those behaviors and we're seeing the benefit of it and then reinvesting and improving the way we work. To me, it's still better than saying, let's spend 6 months upfront planning, have a big guess and hope like hell, you know, that in 2 years' time the guess we had was right and nothing changed in between. We all know that that doesn't work, and especially in the world of data and analytics, it's moving so fast, business is changing so fast, so we know it's going to happen. Yeah.

Excellent.

Alright, well, thank you very much, Sam. It's great to catch up again.

Thanks, Sam. Nice to meet you.

Thank you very much for having me.

We'll catch you soon.

Thanks, guys.

Alrighty, cheers.

And that, data magicians, was another Agile Data podcast. If you'd like to learn more on applying an agile way of working to your data and analytics, head over to agiledata.io.