AgileData #2 - Adopting the AgileBI way with Sam Sewell

Foreign.

Welcome to the Agile Data Podcast where we talk about the merging of Agile and Data Ways of working in a subscription simply magical way.

Excellent. Well, today we've got Sam Sill who's come to join us to have a little chat about the thing we love to call the Agile BI way. Sam, welcome to the Agile Bi Way podcast.

Hi guys, glad to be here.

Hi Sam.

So I think probably to start off, maybe you could just tell us a little bit about your background and what you've been up to for the last 10 or 20 years.

Absolutely. So I've been mainly involved in the data BI reporting world for about 20 years. It was my first actual office job was data entry and from there I moved into can I get a report on the data that I've entered? And somebody said, hey, that's kind of bi. You're now a bi person. And I kind of, I've kind of grown in that way. So I didn't study this at uni or anything. So for around.

10, 20 years I've been involved in this and I don't think I knew I had a profession for probably the first 10 years. It probably wasn't until I met yourself, Shane, back a good long time ago where I started realizing how much depth and how much knowledge and history and how much of a big industry and professional fetch them this was.

And in, in the period I've been working in this space, I think I've touched on a number of different areas of these disciplines. So I've had roles looking after platforms, I've been a consumer, I've been someone who writes reports, I've worked with people doing modeling, but not, probably not smart enough to do it myself. And I've managed teams across a lot of different technologies. So like, I do love what I do and I'm always interested in where it's going.

Oh, that sounds, that sounds great. It sounds very similar to my background actually, Sam, and banking and dealing with data more and more in every role that I moved into until I kind of realized that I was a data person.

So when did you actually hear about this thing called Agile bi and, and when was your, what was your first experience with it?

Well, Agile BI has been really recent for me. I mean, I only started hearing about agile about five years ago. So I've worked in banks, I've worked in some public sector, I've worked in energy sector.

And I think that probably some of the leaders for agile in my understanding in.

Wellington, where we are, were some of the tech companies who were embracing they were small enough and fluid enough to have a good go at making these new ways of working work and be effective. So my introduction to just Agile itself has been very, very slow and gradual. And to be honest with you, I know most about it because my partner worked at one of these tech firms and she embraced it and she learned about it and she blogged about it and she really introduced me to what an effective way of working it can be. It's really being mindful about the way you work was one of the big benefits of Agile. It's working more deliberately. And then Agile BI was something I started hearing about with some of the people we've got in the market here. Some of the luminaries in this country were starting to think about applying Agile and BI together. I encountered it myself a few years ago when I was put in a position to be working in an Agile BI team.

But none of the people had necessarily been agile before or had thought about how you apply BI to Agile. And there was a lot of head scratching and a lot of challenges we knew we had to work through in that space, but we didn't have the answers. But luckily there are some people on the market who are starting to come through with ways of meshing BI and Agile together to make it work.

So when you were looking at that back in the early days, how much content could you find about how to actually apply Agile in a data space?

Next to nothing, Right? Next to nothing. So it really, again, it's been the conversations with you where I've really come across someone who's put some proper thought into it and then has had an opportunity to work with it in the market in real life situations because there's lots of things on paper, on books, there's lots of people thinking about things, but you really need to connect with people who have done it in anger on programs, on projects, in real life situations.

So given that we've just been through a project where we've kind of. Or in a delivery where we were definitely doing it with. Inga, tell us about your experience with some of the patterns that we kind of worked on together and started implementing with the team there.

Yeah. So overall it goes back to me to two things. One, it's taking a new approach because I think for most of the time I've been in bi.

We'Ve known there's challenges about how BI programs and projects run. We know there are delivery challenges, we know there are scope challenges, we know these are some of the more difficult projects that you'll work on. In organizations. So having a new approach, having a new way to look at old problems was really exciting. It gave you a challenge. Okay, well, maybe this is the one, because I've spent 20 years working BI projects where we've not always been as successful as we'd like to be, but maybe this is the one. This is a paradigm shift, a complete different way of thinking and looking at the problem. So that been really exciting. And then because you're doing that, we go back to working deliberately. You're not just, here's some problems, let's try and solve them. You're looking at approaches, you're looking at techniques, you're looking at tactics.

To approach the work to help you be successful. So the work we did recently, I think the way that the approaches, the patterns, the techniques were able to connect with the teams who are going to be applying them, resonate with them, give them hope that you've got a good chance. Because you don't want to start a program unless you really believe you've got a good chance of putting it through. So to be able to put that into people, to be able to bring them on the journey, to say, we're going to do this and we're going to do this, and this is how we're going to do it, and this is why we're doing those techniques, I found, was a really good way of doing it. And the approaches, the patterns, the techniques that we were implementing all seem really solid. They all had good.

Background behind them. They were all there to fix specific challenges that we've all seen before in the past.

So the business, so the product owners that you brought along on the journey with you, what was their sort of reaction? Were they experienced with this previously or was this all new to them? And how did they, how do they sort of like come along with you on the journey?

So a lot of this is new to a lot of people. And in programs you'll often have people come on who aren't necessarily bi or data people, background. So it worked well because all the tendrils start coming together. This is what we're doing, this is why we're doing it, this is how we're going to do it. So we found with the techniques, it was. It was relatively straightforward to bring people on the journey for how we're going to do it and why we're going to do it. Because often people are looking for someone with an approach, with a vision. And this is all part of a vision and an approach and a technique.

It doesn't have Many or any sort of gaps in what you're doing. You can see a path, you can see the way that you run the workshops, you can see the way that you are listening, capture the information and why you capture that information. A requirements document, an old waterfall requirements document. It's got lots in it. I don't know if there's. It's always simple to justify why you're putting all those bits of information, why you're gathering that and putting it in a big document. It's not all going to be needed, but this approach was. Everything had its purpose. I need to ask you this question so that I can make this decision and do this piece of design.

Sounds great. Yeah.

So with the team you were working with, you started tailoring the patterns or the processes, so you kind of started forming different ways of working by using those techniques in ways that I hadn't seen before. Do you want to tell us about some of the things that you saw the team kind of do as part of it?

So, yeah, we found that we would take the ways of working and we had to allow the incumbent team members, the people who worked at that organization, an opportunity to take ownership of those ways of working and invest in them so that they became their own way of working. And when you do that, it allows them also to do some tailoring, to put it into the vernacular of the organisation and help them sell it within the organization. So you take this great way of working and then you put your own organization's branding, ways of thinking, ways of working on top of it. So.

It'S still very much the same underneath, it's still very much the same approach, series of technique, patterns and processes, but it's branded for that organization, for that team. And then you allow the people in the team to make small changes and edits, keeping consistent with the, with the philosophy of it, but in such a way that enables them to sell it and push it and go out there and it really has taken off in the organization. The terminology is the labels for the information products and the labels for the type of workshop and why they have them are being adopted at all levels of the organization.

Because it makes sense to everybody and it resonates with them. So, yeah, it was a very simple set of techniques to apply even to a very large organization.

Yeah, so it's kind of like a best fit for the organization. Not. This is the Agile Manifesto and we're going to follow this to the letter. And we found that in my organization as well, that if you just tailor it a little bit to get a few people across the line, then it's worth it. Definitely.

Yeah. Well, the challenge is buy it so you can have the most amazing approach. And I've seen Agile applied in organizations where it hasn't landed because the people around it. Change is the hardest thing in the world in organisations. Change the way you're thinking, working, come to an acceptance that the way you do your job today is not necessarily the most effective way of doing it. Not everybody's open to that time of change. In all organizations you get people who are career people in the field, but you also get people who are there because it's their day job and they're not necessarily students of what they do for a living. So you've got to be able to excite everybody in the team and help everybody on the team on the journey. But you do need to keep the philosophy, you need to keep the why. Why do we have this way of working? How do we structure our conversations to capture the why? The whole time the same. But you can change sort of the colors and some of the looks and the feels of the thing, but it's the philosophy that's key.

So on that, I mean, often what we find is, especially when we're working to organizations where Agile is new, there's the vision, the desire to behave differently, to work in a different way. And teams of people are normally then mandated or seconded or encouraged or dictated to work in this new way. And for some people it's really uncomfortable, but they make that change. And for some other people it's really uncomfortable and they don't make that change. Did you find that, did you find that given that a team kind of went on this journey, that there were people amongst the team who opted out and how did you deal with that?

Yeah, so. So we found that actually a higher percentage of people were able to pick it up and adopt it than I was expecting. And because it does speak for itself, we're not trying to sell fairy dust and magic. We are. Someone says magic doesn't happen here. Can't remember who said that. Someone I know says that.

On a regular basis. But you're selling something that's, that's got good reasoning behind it. So a small number of people really struggle, but that's an organizational challenge. I guess. Often I found the people that aren't as open to picking up new ways of working will find a role for themselves to still be effective in their role.

I've never found that I've been able to get 100% change everybody on board and you don't want to go in and necessarily change the people in the organization. But I do believe there is a pot for every lid, as it were. So the people that are not able to embrace it, I think it's more of a time and an exposure thing. But in our world there's always work that needs to be done. So it's about harnessing the energy of those who are excited by the change and hope that over time they bring everyone on the journey in a show don't tell way. So they may be skeptics, but if you do things really well, you will bring people on the journey.

That's my hope.

Yeah. So we've found.

That the retrospectives actually expose people's.

Lack of buy in and we've used retrospectives to try and change things to fit everyone into the way of working and get everyone feeling comfortable and having those tough conversations as a after your timebox period is finished and you're looking at improving the next one.

So have you used retrospectives in your workplace and how have they gone down?

Yeah, we absolutely have. I think we probably could have run more. And retros, I think are one of the ceremonies in the agile world that can be run in very different ways and have very different feeling around them. I know my partner, I think, bought a series of flashcards to run retros that gave you lots of different ways to phrase the conversations and get people to interact. So I think the retros take a little while to get, get going, to be effective and I'm not sure I've seen them reach that stage yet where I currently am. I think they exist for a very good reason and it's part of the change to get people to that openness and that honesty, to work together, to not to blame, to not to point fingers, but to collaboratively work on better ways of working moving forward.

Yeah. So I think with retros it's interesting from my point of view, retros tend to come out of using Scrum, but they seem to also have the most available list of games in the marketplace. In terms of making retros fun, there's lots and lots of places you can go to find awesome games for retros. Not so much for backlog, grooming, story planning or any of those other things that come a little bit more boring. But yeah, flashcards and different retro games is kind of exciting.

Yeah, I enjoy them.

So obviously as part of what you did is you've come along on a journey with that current process or project that you're working through. What would you say is kind of the hardest part that you hardest thing you've encountered? Kind of introducing agile ways of working into an organization that's going through massive transformation and is probably going on at the beginning of their agile journey as a business for versus the difficulty of moving to an agile delivery method.

I think if the organization was already there for agile, if I were to work at one of these tech companies that's been doing agile for over a decade, everyone in the organization's bought it.

I'd say the most difficult thing would be transitioning the team and getting them to embrace the new ways of working. But when you're actually part of the vanguard into agile ways of working in your organization, you not only have to change the way the team work, the way your stakeholders work, but also really the way the enterprise operates, the way it thinks about funding deadlines and.

Balancing organizational change. So for me the most difficult thing probably is that wider and senior stakeholder management of setting expectations, of finding space to learn and grow and move while still being able to consistently and constantly deliver to the organizational needs. And I guess that's always been a problem. But.

It'S also fighting the temptation for me to just go, oh okay, fine, I'll just get a whole load of old school developers and we'll just put them in a room and we'll just, because this is really important. So we'll do this bit waterfall just to get it done. So having to fight myself a lot to sort of hold my line, hold my course, to push the change through.

Yeah. And I mean I've found the same thing as I change as I move across organizations and I start moving into a new organization to help coach them or their teams. It took me a while to realize that at the beginning I was focused on agile delivery. I was focused on the team and how the team works and getting the team up to be self organizing and responsible and, and self aware and optimising themselves and bringing that change to that team. But actually there were two things around that that were just as important as agile delivery. There was business agility. So the number of times you work with an organization that talks about a project with a funding for a window, heaven forbid a project manager or program manager in a Gantt chart that sits above an agile way of working. So it's the ability for that organization to become agile and change the way it operates. And then the second part was kind of after the Agile team is delivered is getting everything into front of the user. So automating the deployment, automating the operations, stopping. The problem of as the team delivered more and more, they were often held responsible for maintaining it. And that wasn't automated. So we saw kind of drain, right? We saw.

The team's velocity for new stuff kind of die as the, the bau bleed, I call it happened. So for me, yeah, we focused on agile delivery to begin with because that's the problem we wanted to solve. But now actually it's realizing that business agility and automation of the delivery and ongoing management of it is just as important. I mean, you probably found that as well, Blair, in terms of going from the early days where the teams were just rocking it to now you've got stuff to maintain because it's the same team.

Absolutely, yes. And you've got to balance that with the new and shiny stuff we've found. And the product owners haven't, I've found in our, in our setup, they haven't let go of the product. So they still feel a sense of ownership because they went through the whole thing with you and.

Some of their refinements. We've had to squeeze into building new stuff and it's just that whole squeeze. But yeah, the thing I've noticed is that ownership aspect, that they don't just go with you on the journey and then just go, boom, thanks for coming. You've built this great thing, but that's.

Great as well, right? Because now you've got somebody who actually owns the product, right? Somebody's going to fight for adding to it. It's not going to sit on the shelf. It's not like when we used to buy enterprise software, right? You go buy it, sit on the shelf, you come back a year later and nobody's used it. Wow, that wasn't fair. But, you know, then you've got that adoption problem, right? As you get more and more product owners that want more and more done, you know, and you've maxed your velocity. How do you scale, right. Or do you prioritize and actually on that. So, you know, one of the things interesting for me was the number of information products that you actually defined up front as part of your roadmap and the fact of, because of the type of work you were working on, you know, you had multiple product owners, each with a relatively heavy list of information products to be delivered. How did you manage the prioritisation across that nightmare?

It's difficult. Yeah, it's difficult. So to your point, a few seconds ago, the place I'm at, I don't think we've reached that crunch. Point where you're trying to balance delivery with the support. And I think that's a challenge I know I've got coming up, which I'm excited about tackling.

With the IPs. So we've gathered a good few hundred IPs and there was a real burst of excitement initially with the IPs, because again, you're giving the business a good vernacular. One of the ways that I describe the BI way, the Agile ways of working, is it's a common language between the three groups that you're working with, between the people in the business who want the outcome, the analysts who are there to facilitate that outcome, and the people in the information or data teams who are the custodians and the preparers of the information. It gives all of those three groups a way of speaking to each other that makes sense to everybody, which I hadn't seen before. You're always talking different languages. This gave it a conform language that everybody understood. So there was huge excitement around the ip. Everybody wanted an ip, everybody wanted a workshop, everybody wanted a session and we flood in lots of these. One of the challenges was probably consistency and quality, because it was so exciting to go out and run as many workshops as you could, is how do you ensure ongoing that you're working in a consistent way, which is part of why you have the workshops, is to work consistently, consistently, but you can still have different people interpreting in different ways. So that was one of our challenge prioritization.

I think, along with a lot of Agile ways of working. We have to really road test it to see if it is successful. Transparency and visibility are one of the keys that we've been trying to work on, so the whole enterprise can understand what is on the backlog and can understand the import of everything on the backlog. And then it's finding the right group of stakeholders about the right level in the organisation who can collaboratively work together to balance that priority. But that's still, I think, a work in progress for us. We definitely are mindful of it and we're definitely getting the conversation rolling around the organisation and in the governance groups to let people know that they need to be aware this is working. We have our backlog, we have it in Agile Tool in jira, so it is very accessible. We keep it up to date. But how that journey is going to go over the next year for how we work through that, I think that's something I'll come back and tell you. And I think one of the things.

That you did that I really liked was yes, you've got your backlog of Jira and you've got boards and it's visible. But in my opinion, Jira is not the easiest product to look at, understand? So one of the things I really liked was you almost took an infographic approach to creating some really high level infographics around the IPs, the backlog ways or lenses of looking at it to say maybe by kind of event or by subject area or by you put some lenses on there. That explained the different ways the backlog could be viewed. How do you find that in terms of did that effort actually come up with value? Did the stakeholders actually look at it and understand a little bit more how big the elephant was the different ways that they could prioritize what was first and what was second?

Yeah, absolutely. And it's something I try and apply whenever I'm working with BI teams is to use bi. We are the BI people, so. So we should be using our own techniques and tools internally, BI on bi. So creating infographics, being able to pictorially demonstrate, if we do A, that unlocks B, which unlocks C, D and E, put it in colors, patterns, shapes that are accessible to everybody, that people can put in their hands. As much as we're digital, people still want things in their hands, in their book that they can open and say, well, this is where we are, this is what we're working on. So being able to give people pictorial aids that show the different lenses, as he said, around subject area.

Around if we. What unlocks other things. Because at the start of a warehousing project, you have nothing. And you can't get right down to your nuanced products until you've done all the foundational work to, to show how all of that stacks up and adds up to grow something that's greater than all the component pieces in a way that resonates with people we found really effective. So I think if we hadn't done that, life would be a lot more difficult for us. It's been a really good sales tool for the team and what they're doing.

So I think also for me, when I do my personal retrospective around the Agile BI way, one of the patterns that, that we kind of came up with during that process. And it comes back to your comment around scaling the facilitation of the information products as people kind of do it the first time and you get more than one or two people kind of running out and trying to scale that facilitation.

The quality of what they capture becomes slightly variable and I think in hindsight it wasn't until we went into the sizing workshops for each of the IP where we actually sat down together with the team and actually talked through each of the information products that really the gaps started to become visible and the people that gathered them realised where the focus should have been because they knew actually got more of an insight into what the outcome is. So I think for me, extending that pattern out to, if we're bringing new teams on, go show them how to do the IP facilitation work through the first couple with them, let them go out on their own, but very quickly bring it back to a sizing workshop because that by talking through it and people who weren't involved in the facilitation actually being able to ask those questions that I think helped refine the process for the people that went in and gathered it made them better. So if you look at it from that point of view, with the next piece of work you're going to do, when you're going to bring the Agile BI away to another team, what do you reckon the one thing you'll probably focus on improving next will be? Where's the gap? Where's the thing you're going to bring to the world as one of the patterns that the rest of the Agile B waysts, or whatever we're going to call them can leverage?

That's a good question.

And not one.

That we wrote down for you. So you're on the spot and it.

Gets us all to think of.

Yeah.

Bribery and threats I think is something I'd like to utilise more in the workplace, but I think there's, I think very much a show, don't tell very early on I think is going to be maybe an approach to take. So as soon as you've got enough arms and legs and heads in a room that you can actually run through these techniques and get people access to a system where they can put them to practice. I think being able to create a really transparent use case where you can say this is what we did on this date, this is what we got, look at what we can do. Because a lot of the experience in the BI way was selling the value of it and a lot of people grabbing onto that and really understanding it, but for the wider organization there was that it wasn't visible, it wasn't transparent. So being able to actually do an early proof of concept to go all the way through and prove the doubters around you and get people on board, I think is what I'd like to do because we tried to do that where we are, but.

I think probably could have pushed to do something faster earlier.

But do you think that's because although the organization said they wanted to be agile, they really weren't business agile. And so there was an outcome. There was a number of information products or dashboards or reports or data sets or API. There were a whole lot of things that needed to be delivered. And it's just the fact that you and the team decided you want to take an agile BI approach to deliver them in a better way. Therefore actually a lot of your time was selling to the organization why this approach is actually valuable, why you're investing in these teams doing things slightly differently.

Yeah.

Do you think that's kind of why that was the focus area for you next?

Certainly, yeah. I mean a lot of energy goes into that sales work. So if you could, I mean you could almost deliver two different ways at once right. Early on to make it really, really clearly articulate the value of the ways of working working. But yeah, that overhead of selling the techniques to the stakeholders was where a lot of the energy goes. If you can get someone to do that work for you. So you and the team can just plow through the work.

Yeah, we've found that as well with the concept of the MVP in front of stakeholders early on gives them something that they can report back to their, to their managers and it's actually becoming a thing is quite exciting. And yeah, I think I'll be working on the MVP a lot closer next time too.

So we've talked about MVP ip.

Yeah.

And that was one thing I saw at a site I was at previously where they were able to get one number, one number all the way through. And they weren't using a true agile BI way of working. They were doing traditional BI and trying to apply Agile principles to it.

But they were able to take one number all the way through and put it on a dashboard. So it was a dashboard of nothing except one number. But they were able to get that number through in one sprint.

Yep. And we always say, you know, vertical slice, thin slice all the way through if you can, is is what we should do. Right. Sometimes for 101 reasons. It's not an approach we can take initially, but if you can thin slice it all the way through. And the other one, I think picking up on your point, we talk about being ad hoc, we talk about doing agile and we talk about being agile, but it's a journey. I'd rather hear the words we weren't really being agile. If we came in and somebody audited us of our behaviors and the way we working, we're not going to come through with the retrospective would have lots of things we could do better. But we're starting that change, we're starting to adopt those behaviors and we're seeing the benefit of it. And then reinvesting and improving. The way we work, to me is still better than saying let's spend six months up front planning, have a big guess, and hope like hell that in two years time the guess we had was right and nothing changed in between. Because we all know that that doesn't work. And especially in the world of data and analytics, it's moving so fast, business is changing so fast. So we know it's going to happen.

Yeah. Excellent.

All right, well, thank you very much, Sam. It's great to catch up again.

Thanks, Sam. Nice to meet you.

Thank you very much for having me.

And we'll catch you soon.

Thanks, guys.

Alrighty.

Cheers.

And that Data Magicians was another agile data podcast. If you'd like to learn more on applying an agile way of working to your data and analytics, head over to agiledata IO.