#0020 - Scott Ambler - disciplined agile toolkit

This is the no Nonsense Agile Podcast. Join us for discussions on agile product development and leadership with world class experts who provide valuable insights and practical advice to industry professionals. Subscribe now to learn the latest trends and best ideas in the field.

In this episode we talked to Scott Ambler about the Disciplined Agile Toolkit. We discussed toolkits vs frameworks, tailoring your ways of working for your situation vs prescriptive methods, method prisons and framework jails problem with two day mastery certifications and the over commercialization of Scrum and safe. And finally we talk about the PMI and Agile and products versus projects. Join us for a fascinating discussion about Disciplined Agile delivery.

Welcome to the no Nonsense Agile Podcast. I'm Shane Gibson.

And I'm Murray Robinson.

And I'm Scott Ambler.

Hi Scott, thanks for coming on. So today we want to talk to you about the Disciplined Agile Toolkit. So could you kick us off with a brief introduction?

Yeah. So I'm one of the co creators of the Discipline Agile Toolkit. I am also the thought leader behind the Agile Modeling and Agile data methods.

So what is Disciplined Agile Delivery?

So the Discipline Agile Toolkit. Well first it's not a framework. The frameworks and methods tend to tell you what to do. They present what they call best practices. In DAE, we don't do that. So instead we focus on here are the issues you need to think about and here are some options. We can't tell you what the best practice is because we don't know your situation. But what we can do is to help you pick the right strategy for the situation that you face. It's more around helping you to get better at getting better rather than telling you what to do. It originally started around 2008, 2009 timeframe. So I was working for IBM Rational at the time and myself, my team and business partners that we were working with were going out to organizations and in helping them to apply Agilent Lean strategies at scale. But anyways, what happened was after a couple of years of working with organizations, a bunch of us started noticing several things. First, everybody was struggling with this crazy Agilent Lean stuff. They were spending a lot of time and money trying to apply it in their situations and they were all doing it differently. So observation one they need some sort of process help. Observation two, they don't need a prescriptive method which is what everybody else is doing. And instead of telling people what to do, we realized we really needed to teach them what to think about and how to make better choices and how to improve. And that observation is what then led us to develop Dispenager Delivery, which evolved into what's now the disposal toolkit.

Yeah, I read Disciplined Agile Delivery, and it read very much like a menu of options for different parts of the system development lifecycle if you're doing it in an agile way. I agree with you that a lot of the process frameworks are very prescriptive. I've seen Ivar Jacobson, one of the original RUP guys, saying people are stuck in method prison and he wants them to get out of it.

Yes, it's a fantastic observation. So the basic idea is that when you successfully adopt a framework or a method, you hit the limits of it. Scrum solves a certain problem, SAFE solves a certain problem, and so on. But they don't really give you any advice for how do you go beyond the method. And frankly, they don't want you to move away from the framework that would ruin their business model. This is why there's not a lot of advice. Not only how do you do software engineering, which is what this financial delivery was all about, but what about the rest of DevOps? What about value streams? It doesn't really matter how effective your software teams are if they've got an anchor on their neck, or if the procurement people aren't very effective or if the HR people aren't very effective. A fleet travels at the speed of its slowest ship. So you really need to focus on enterprise agility and how do you improve across the organization.

Shane and I are big fans of tailoring. We will often say that you need to start small, find out what works, and grow from there because each organization and situation is different.

Yeah, and funny enough, I think I got it from Scott. When I started my Agile journey, I stumbled across your agiledata.org site. And so for me, that philosophy of a toolkit of taking practices and patterns and crafting your own way of working, that stuck with me for the last 10 years. And it's how I coach teams. If I compare dad to Safe, you had a toolkit, you had a bunch of practices and patterns that you had adapted from other things and said, if this looks like the thing you're struggling with, try this. Whereas Safe says, here's all the things other people do, you shall do, it comes back to that fundamental philosophy of menu versus prescription. Thou shalt do it this way, or here's a whole lot of choices you have. Try them and then craft your own way of working. Craft your good practice, because you're the same as everybody else, but you're also a little different.

So we both really like the menu driven tailoring approach. And I am a SAFE SPC and it's very prescriptive. So with safe, you get an hour by hour description of what you've got to do during PI planning, like every meeting, who's got to be there, what you do, and so on. And I find that with safe, people spend two months planning for their PI, then they go through it, they expect the teams to commit to everything in detail and managers expect them to then deliver it. Exactly. So I know SAFE is not supposed to be fixed scope, but that always seems to be the way it's implemented.

Yeah, to be fair to the SAFE guys, SAFE can be an improvement compared with the traditional ways of working, but it's only a start and I think that's where people fall down. We see a lot of SAFE shops that are in fact doing better than they were before. You got to give them credit for that. But there's still a lot of room for improvement. And this is where the DA toolkit kicks in. We get you going, but then as you get comfortable at this level, start tweaking the dials and improve your way of working based on your situation, your people, the challenges that you currently face. So it's not just here's a bunch of best practices, go for it. It's, yeah, here's a good starting point and here's how you improve from there on what you need to improve on. So it's a much different mindset.

Yeah. I think though, we gotta talk about the elephant in the room here, and that is the pmi, because a lot of people in the Agile space have a negative experience with project managers. Now I've been a project manager myself, so I don't view them so negatively. But when I was a member of the pmi, I met a lot of very old fashioned conservative project managers at PMI meetups.

Yeah, fair enough. I was skeptical too. But one of the things that attracted us to PMI was they were actively reinventing themselves. So we got to see under the covers and we also got to see some of the work in progress, in particular the PMBOK Guide 7, which is now principles based. There's still a little bit of prescription, nowhere near what it used to be. Still some traditional stuff and some agile stuff and lean stuff as well. Much better aligned with our philosophy in daa. For example, in daa, it's not just agile, so we put techniques from the Agile space into context, but also lean strategies and traditional strategies as well. Because Our focus is on what's the best you can do in the situation that you face and how can you get better. And sometimes the best strategy for the situation is a fairly traditional technique, and that's okay. And PMI is still a lot of legacy people, still coming up to speed. There's also an issue around tangible versus intangible efforts. So the intangible space is working with debts, so your IT projects or media projects, artistic things, creative things. And then there's the tangible space where you're dealing with atoms for the most part. So you're building bridges and houses and whatever. And those are two very different things. Working with atoms, you're starting to deal with laws of physics and stuff like that. Working with bits, things change instantly. So you need to work in much different ways in those spaces. The more traditional community that focuses on tangible efforts can certainly become more agile. Like applying agile in the construction industry is definitely a thing. And to be fair, applying traditional strategies in the intangible space do apply as well, because we have to respect the fact that the traditional folks have gotten a few things right. But certainly the PMI has been evolving just like every other organization. And if you look at the latest materials, I think you'll be impressed. The principles based approach in the new version of the PMBOK guide is worth looking at.

Sounds a bit like PMI have become agile in terms of inspecting, adapting and changing who they are due to the changes that are happening around them.

Yeah, and they have to. It's not something they could possibly avoid. Agile is real. It's here to stay, like it or not. And one of the things that's interesting about the DA Toolkit is it goes far beyond software. So I think one of the challenges with the agile community is it's still overly focused on software because it's history. But finance and procurement and legal and marketing, all these other aspects of your organization are not IT specific. Marketing and construction firms can improve. And even construction firms are still effectively software companies. They're still running on software. They've got logistics software, they got design software. So the key things in a construction company are often software based. I don't know of any construction companies that compete on their ability to pour cement. It's not the way it works, but they darn well better be good at pouring cement.

Don't get me wrong, PMI has traditionally locked all of this stuff down behind firewalls, which is a problem for people trying to learn about it. Is that the case with all this DA Flex stuff?

It's Pretty much all threeware. Other than what happens in Courseware, of course, it's either completely publicly available or behind a member part of the firewall, but not the paid firewall. The only thing you need to pay for would be the choose your wow book. If you want a paper copy as a member, you can get a PDF copy for free. And the choose your wild book is 20 bucks.

So have you found that by moving dad under PMI that it's opened up the audience to your toolkit?

Yeah. So we were being out marketed. There was nothing we could do. We were overwhelmed. The Safe guys are marketing professionals. The Scrum folks just had word of mouth, totally entrenched. We weren't getting our message out. So PMI has a lot more marketing cloud. We've got the chapters, which are amazing. They really are there to help and to share information to help people come along. So we've been getting the word out of the chapters and we do have a marketing budget now. We just didn't have the marketing budget back in the day.

Near the beginning, you talked about your journey. You were looking at. There was lots of waste in the way organizations work. Do you think that's any different since you started? Has it gotten any better?

No, I don't think it has. Maybe a little bit. But the challenge is people are overwhelmed with the marketing rhetoric. And even though I love Scrum, I like to talk about how people have Scrum blinders on. People have gotten roped in by the terminology and they focus just on the Scrum stuff, not realizing there's this wealth of material out there available to you. Like earlier you're talking with the Agile modeling site, which is fundamental concepts there. But if you don't know how to search on terms like modeling, you'll never find this material if you even think to go look for it. And that's a serious problem. So you get people just locked in to Scrum or SAFE because that's the only thing they know. And then they get locked into Framework jail. Not only are they in Framework Jail, but they can't even recognize they're in jail. And so then the idea of breaking out of jail is a bit radical. I was just on a call earlier today where people were equating Scrum with Agile and they had no concept that there was other things out there. And it's amazing how many people have never heard of things like extreme programming or tdd.

My view is certification or training is great. Starting off with somebody explaining the basics to you gives you a language, gives you a place to start. It's valuable, but it doesn't make you an expert. And it's not until you have experienced and practiced with teams doing it for a while that you realize that there are other ways of mixing it up.

I'm a firm believer in good training, coaching and learning, but you're not a master after two days. But it's the reality is the marketplace. Unfortunately, if you try to force people to earn their certs, it's a killer in today's marketplace when you're competing against Take this two day course and you're a certified master versus work really hard, take a really hard test and then maybe you'll pass to become a novice. From a marketing point of view, it's not the greatest strategy. What's also unfortunate, the vast majority of introduction to Agile is this certified Scrum Master training. It's really hard to find a general Agile workshop and then if you do, it's not well attended because you're not getting some mastery certification. So the market's a mess as a result of this. People should stand up and say something. They really should push back.

Martin Fowler had a Great talk at 2018 at the Australian Agile Conference about the Agile Industrial complex. And he was saying that the big problem in the Agile community now is fake agile. Everywhere he goes, people say we're doing agile, we know all about agile. And he finds out that they're doing fake agile. Agile bullshit, frankly. And I've seen that as well a lot when I've coached people. So the two patterns that I see a lot of are doing Scrum in silos with stages and gates, and the other pattern is safe, which I don't think is actually agile at all. It's got the word agile in its name. I don't think it fits with the agile values. So what I see a lot in safe is fixed scope, fixed time, fixed budget, completely disempowered teams.

Yeah, I see that a lot. In the beginning, everybody was trying to push the state of the art forward and there was a lot of really good vibes and we were really trying to do some really good stuff. At the very beginning, Extreme programming ruled the roost. Scrum was absolutely dead in the water. Nobody cared. Agile modeling came out of extreme programming, then Agile data came out of Agile modeling because we were working on fundamental challenges at the time. But then the Scrum certification stuff came around and it was all over. A group of us pushed back hard, but another group of us saw money and money won. And an unfortunate side effect now is that we're all in scrum jail. We've done ourselves a great disservice as a community. We have completely lost the idea of having to work for a certification. If you ever get a chance to talk with real professionals like lawyers or architects or doctors, and then explain how people learn and get certified in our community, they'll think you're joking. And I can't see how we're going to dig our way out of it. It's just too systemic now.

So you wrote about agile in the data space, which still is unusual. There is very few people that are taking agile practices and applying them in the data space and then sharing the results of what they learned with the rest of us. Do you think we're going to go move towards toolkits that are domain specific?

I sort of hope that's the way it'll go. I've seen some good stuff. There's the agile marketing movement, there's the lean agile procurement stuff. But the problem is they see themselves at the center of the universe. And this is the problem with the traditional data community. They think that they're the center of the universe because they have a death lock on the data. And that was one of the reasons why a bunch of us put the agile data method together to give them a slap upside the head and say, you know what, you can do a lot better than what you're doing and here's how. And then we just called BS on the DM box assumptions. And then in the agile data world we said let's question the assumptions. One of the assumptions in data was that production databases are hard to change. There is no proof of that. And there is significant proof that it's absolutely trivial to change a production database schema safely if you know what you're doing. And that's where it all falls apart. But the problem is was they never broke out of their assumptions. So if we go for domain specific agility, and I think that's a trend, then you're going to start to see silos again because everybody will go off in their own direction. It'd be great stuff by itself, but if it doesn't fit in with everything else, then it's not going to happen.

What I see is the whole idea of moving from projects to products. I really like the idea of having a product focus, you know, this idea of bringing everybody together around the cross functional product team, working iteratively and incrementally around a product focused on real users and real customers.

Yeah, I do too, but it depends. That's a trend that makes sense in the IT world. But the reality is there's always going to be projects. There will always be some sort of agile approach and some sort of traditional approach as well. So I view the project, the product movement, as a subset of the overall environment. So in the intangible space, when you are actually working on products or services, then having a long standing team makes sense. Having them as cross functional as possible makes sense, but they'll never be fully autonomous. I think that agile teams are autonomous is one of the great myths of the agile community. I ran a research study on this a while back and something like 96% of agile teams are not autonomous. You got to collaborate with other teams. You go to the finance folks for funding, for example, you go to HR folks to help you hire somebody. So you might be autonomous sometimes, but other times you've got to collaborate, and rightfully so. So you'll never be fully cross functional. And even if you are, all you got to do is lose one person on your team and suddenly you've lost some skills and you're no longer as cross functional as you were. We need to be realistic about that. But certainly as a goal, as something I want to aim for, yeah, I want to be as autonomous as possible. So in DA Dispenagile we talk about teams being semi autonomous. I was interacting with somebody yesterday at PMI and they're saying, don't you want to bring this functionality in your team? I said, heck no, you're good at that. We just don't want to be involved, basically. Why would we? You're doing a great job, we'll figure out how we can work together and that's it. So in DA we support multiple life cycles. We don't inflict one way of working on anybody. So we have an agile project life cycle, but we also have an agile continuous delivery life cycle, more of a devopsy type of thing. We have a lean project lifecycle and a lean continuous delivery life cycle, like a pure devopsy type of thing. We also have an exploratory life cycle for when you're taking a hypothesis driven approach with MVPs, basic lean startup, and of course a program lifecycle. And it's not released yet, but we're going to be adding citizen developer lifecycles as well. So how do you bring the low code, no code movement with business people building their own systems? So that's called citizen developer. So how do you bring development tools and techniques down to a certain level, but at the same time Most importantly, how do you detect when not to do that? Because there's a significant judgment skill there of, ooh, you're in over your head, you really need some help, or ooh, you should not be touching that at all. So that's a movement and I'm hoping finally that the citizen developer stuff will take off. Yeah.

Whenever people say to me, low code, no code won't work, I keep telling them that Excel's still the number one BI tool in the world because it's a low code environment. You haven't managed to kill it yet with all your data engineers. So on there though, you mentioned something before and you reiterated here, it's this idea of a menu versus prescription. And one of the things I did struggle with the DA stuff was there was lots of choice and, and often found that I got lost.

Yeah. So I think we really need something in between because people need recipes. So if you don't know how to cook, if you don't know what the ingredients are, me just telling you, yeah, go to the grocery store and buy some ingredients and make dinner tonight. Good luck. It's not gonna work out well, so people want recipes or they'll end up going to McDonald's. But the problem is you can't eat a Big Mac for dinner every single night. It's not healthy. You need a little bit of help learning how to cook and you'll never be a chef maybe, but everybody can learn how to cook and everybody can learn how to follow recipes. And even when you start cooking a dish, for the first couple times I follow a recipe and then I start tweaking it because I know what my daughter likes, what my wife likes, and so on. And eventually you gain the skills to put together on your own and you're missing an ingredient, so you throw something else in and you learn those skills. So DA when it was originally presented was here's all the ingredients, here's the trade offs, here's when you would use each ingredient. But we didn't have any recipes. The issue at the other end of the spectrum with the prescriptive frameworks is here's a recipe for chicken cacciatore and you would darn well better want to eat chicken cacciatore for every meal the rest of your life. Because that's the only thing we're going to teach you because that's the best practice. We need something in between. So we are working on starter kits to get you going on that.

I'm wondering if we should do a bit of a summary of our thoughts on this. Shane, what do you reckon?

Yep, I'd go for it.

So I really liked the wow book. I read through it and I really like the discussion about method prison. So many people I come across think that Agile is Scrum and Scrum is great, but it's just a slice of the pie, not the whole pie. And Safe is not the low process framework that we were looking for when we developed Agile. It's a heavy process framework, so people get locked in that and people really need to think about this. Are you in the Scrum prison or the safe prison? Because it's time to break out. There's so much more. I think those of us who were in it at the early days with XP and everything realized this. And the people who've just come in through the Scrum door or the safe door just think that's all there is. And there's also this really fundamental problem going on where organizations are tailoring Scrum and Safe to be like their traditional authoritarian command and control siloed bureaucracy. And to really get the benefits of all this stuff we're talking about, you need to challenge these silos. Factories are great for producing a billion pins that are exactly the same because it's cheap and efficient. But software and products are not like that at all because there's considerable uncertainty and learning required. Every time you write a piece of software, you're learning something new about the business and is it technology? So you're doing something different. And traditional fixed scope, siloed methods work very poorly where there's a lot of uncertainty in learning. And Agile works great. So if you want the benefits of Agile, you gotta do it properly and not trying to force it into this factory mindset.

Yeah. So same key thing came out. It's not a framework, it's a toolkit. Toolkits are where the value are. There is no such thing as best practice. There's just a bunch of good practices other people have done that may have value for you. So adopt them where it makes sense and adapt them where you need to change it and trash them where it doesn't work for you. As long as you're proved it didn't work for you. They just try it for 10 seconds. The idea of framework jail or method jail or framework prison is what I'm going to go and use and abuse because I like it. The idea of tangible versus intangible, that's a new one for me as well. This idea of building physical things versus building intangible things. I always struggled with Describing where waterfall or traditional ways of working had value. I think that tangible versus intangible gives me some quite nice framing of how to describe that. I think the idea that when we have a menu of good practice, I think that's highly valuable. I think the last one for me is our medical or our accounting brethrens, they have a true certification process where they they are deemed worthy to do the work they do. We don't, we use their terms and then we abuse it. So we should look at ourselves and do it properly. So certification has value, but let's not abuse it as much as we do now. So yeah, I think for me, ending out it's not a framework, it's a toolkit that's the same.

So I've been looking for best practices for decades. I've never found one. I've had thousands of strategies and practices presented to me as best practices. But every single practice I've ever run into works well in some situations, so it's best in that context, but is a phenomenally bad idea in other contexts, so it's not really a best practice. Similarly, I've never seen a worse practice and I've looked for those too. But no matter how bad the practice is, I've never found one that I couldn't find something good to say about. There has been some situations where I would recommend that's probably your best bet in that situation, even though I hate that practice, but it's your best bet there. So you've got to be able to choose, you got to learn how to choose and otherwise you're a one trick pony. And that I think is a big problem with a lot of people right now that are effectively one trick ponies and don't realize it.

To reiterate that the context of when that practice or that pattern was applied is important. We all work into work other people have done and we like to bag it. What we hardly ever do is spend time understanding the context of why that practice or pattern was implemented. And once you understand the context, 9 times out of 10 I sit there going, yeah, don't like the decision. But if I was in your shoes at that time with that context, yeah, I probably would have made exactly the same decisions.

So yeah, it's been great.

Excellent.

All right, on that note, we'll catch you later.

That was the no Nonsense Agile podcast from Murray Robinson and Shane Gibson. If you'd like help create high value digital products and services, contact Murray Evolve Co. That's Evolve with a zero. Thanks for listening.