Agile Engineering Maturity Model with Rudolf Groetz & David Heitzinger

Welcome to the Test Guild Automation Podcast where we all get together to learn more about automation and software testing with your host, Joe Calantonio.

Hey, it's Joe. And welcome to another episode of the Test Guild Automation Podcast. Today we'll be talking with Rudolph and David all about their Agile Engineering Maturity Model and probably all other kinds of topics that come out of this. But really excited to talk to them today about this model. Rudolph has over 30 years of experience in IT and is a passionate software tester. He works as an Agile Engineering coach at his company's test and test automation department and I love his motto. His motto is Test automation is not an act. Test automation is a habit. Love it. In addition to professional articles and various magazines, he also provides the community with conference presentations and organizes the Vienna Agile Test Automation Meetup that has over 1,000 members. We're also joined today by David and David is the head of Agile Engineering Support and he has extensive experience as a software developer and architect. So together with his long engagements in agile topics, all which are really key agreements to tackle transformations that we're going to be covering using the Agile Engineering Maturity Model. If you're looking for ways to help your company and agile coaches transform your organizations, you don't want to miss this episode. Check it out.

The Test Guild Automation Podcast is sponsored by the fantastic folks at Sauce Labs. Their cloud based test platform helps ensure your favorite mobile apps and websites work flawlessly on every browser, operating system and device. Get a free trial, visit testguild.com sourcelabs and click on the exclusive sponsor section to try it for free for 14 days. Check it out.

Hey guys, welcome to the Guild.

Hi Joe. Thanks for having us.

Hi. Hello. Good evening.

Hey David. Hey Rudolph. Great to have you both. I guess. Before we get into it, is there anything I missed in your BIOS that you want the Guild to know more about? David, let's start with you.

Yeah, thank you. I already mentioned the most important stuff name is David, responsible for engineering in our company. And at the moment we are in the middle of an engineering transformation and we have a lot to do in terms of test automation testing test strategy. It's a huge move that we are planning and that we are doing from really classical software development, life cycle, classic testing to really come into this DevOps setting.

Awesome. And Rudolf.

Yes. So as you already mentioned, I'm an Agile engineering coach in David's department. In addition to that, I'm also the Guild lead for our test and test automation guilds. So now maybe you think so. Ah, they are Working according to the Spotify model. So no, we do not call it Spotify model, we call it adaptive organization.

So why don't you use the Spotify model then? What have you done that makes it different?

It is more as this Spotify model and to see the difference is. So the Spotify model only works for Spotify and not for another company. And this is why we said so it is better that we call it the adaptive organization and it means also that we permanently try to improve our product teams and so on.

If you call it adaptive, you have a loophole that you also can do some classic stuff on the side.

That's true. So David, you mentioned your organization is going through a transformation and it sounds like you guys are really in the middle of it now. I guess before actually we dive in there, I mentioned the Agile Engineering maturity model few times. Is this your model that you all came up with or at a high level? What is this model?

Maybe I start with a few sentences about the why? As you said, we are already quite far with our Agile transformation and we realized immediately that we also need to invest on the engineering side because you cannot deliver fast and frequently if you don't have your engineering right. But when we realized it, we also said okay, we need to know where to start. And, and this was the starting point that we said that we need a tool where we can assess where are the hotspots and also let's try to quantify the measures. We have to quantify the invest. We need a roadmap and this was the start.

What is the background for this model? Yes, it is our own model, but we started to take a look to grab something from the TPI model, the TMI and so on because so it started more from the test and test automation perspective. Then we have also to cover the DevOps part and then we brought. There is the. And then we have seen other models. One is from Adidas, I think so. And one is from another banking from another bank. And then we used from each of these models different approaches. We then created our own questions and in the meanwhile we have the second edition in place and currently we are working on the third edition because we are adaptive.

Very cool. So when you say adaptive, is it based on a Sprint or like every Sprint you get feedback and based on that feedback you build it back into your model.

No, no, it is not according to screens. So every time when we come back from a maturity model from an interview with one of our teams, then we are sitting together with the other HR engineering coaches. We discuss the outcome of this maturity model and most of the time one coach then says so ah, wait, so there is one question which we should also bring into our model. So let's discuss this in next time in our retrospective.

Nice. So if someone's thinking, trying to get their heads around what this model is, what are the pieces of the model for someone to be able to implement it? So then we could dive into maybe each area.

Actually it's quite simple. It's an Excel sheet with questions, but a lot of them around 150 questions. And these questions are from different areas we have clustered into CI cd.

Into test general, into test automation and test. And these questions come always in three categories. The first category is the crawl. The first level, this is where we say okay, this is a must have a product must do it this way. The second level is the walk where we say this is the should be that the product should do it like this. And then the third level is the run level when you're really top notch and you really do the nice things. And we decided to go for an interview, a self assessment with an interview you could say we sit together with the team, go through the questions. The question says yes, no other team says yes, no, yes, no. And that's it. We have experimented or tried out self assessment using some web formula or doing it, we were thinking about automatically deriving it out of the systems. But in the end this situation of the dialogue of sitting together, of discussing the questions proved a really, really good element of this tool because so this is the way how we get feedback and this is the way how we can transport our ambition. And this proved to be a very beneficial side effect. Through this discussion we can tell the people what is behind it. Why do we want, I don't know, explanatory testing and this communication, this is really something that proved really worthwhile and helped us in spreading out our goals, our knowledge. And I would value it even more important than like really the outcome in terms of figures and measurability and that we know where we are really. This communication was very great for us to experience it.

But there is another important thing. Yeah so it do not only to foster the communication between, between the product team or the feature team and the agile coaches or the interviewer. What is also important, what we have seen that it foster the collaboration within the team. So it could be that one of the developers says ah, we are doing that. And another developer said wait, wait, wait, wait, this is not what I think what we are doing. And maybe we should discuss this.

Nice. So I guess what I like about this model is you work for a large organization, a bank. So I assume what it is. You send out these questions, all the different teams, and all different teams are at different levels. And then based on that input, your coaches then know, oh, we need to focus in on this for this team or this other thing for this other team. Is that how it works?

Yes, exactly. We are like visiting all the teams have a session and the outcome of this questionnaire we share with the team. And we also come up with a bit of a summary where we are proposing measures. We also summarize our findings and in the second round we discussed with the team and sometimes also with the stakeholders, like the product owner, some other stakeholders, what measures make sense. In the end, it's the responsibility of the team. Finally the product owner has to decide which measures make sense. But this is then the outcome that the teams know the hot topics, where they should invest. And naturally we are also then here to help them, to give them advice, work together with them.

Nice. So Rudolf, how does this work in the real world? So you send out the questionnaire, you get feedback, you understand a team's need, you send your coaches out to help team them, they come up with measures that they'll use. Is the measures then what you use going forward to assess a maturity of each team.

So first we do not send it out and get it back in the meanwhile, it is not that we initiate this interview in the meanwhile, it is that the product teams come to us and says ah, hey, so can we do an intuitive assessment because we want to know where we have to improve. And then we discussing this in the team with the HR engineering coaches. Then we try to find coaches which have experience, maybe from former work together with them that we say, ah, this coach knows about this team because he worked for that. Then we have a meeting, of course remotely now and then we discussing the Excel sheet. Then one day later we discussing the outcome in the interviewer team and three or four days later we send it back to the team that they discuss this. And sometimes then the teams are immediately jump into we want to go to improve our test automation. So can you support us there? And then from the coaching perspective we are asking so how we can support you with teaching with upskilling. What we are not doing is that we are a resource pool that someone can say, so I need a test automation engineer for 20 days. This doesn't work. But what we are doing is. So it is also also possible that one Agile engineering coach goes into, the team is working together with their test automation engineer. And this has always end date. That means after six to eight weeks the desk automation knows the Agile engineering coach is stepping back and then I am alone. That means that he has all the things to do, that he get all this information and the support and the help from, from the coach. And in the meanwhile we have I think two to three teams where we had the segment review where we then discuss all the improvements they did.

So David, being the head of Agile Engineering, how do you get people to embrace this? It seems like you already have teams that actually reaching out to you. But how do you get teams and your company really to embrace this, make sure that the survey is being filled out and the teams really are getting benefit from it.

I think this is the most difficult thing what you are addressing here because this is a cultural change because what we are actually telling the people is you have to do it differently. Then you have teams who are together already for more than 10 years, 15 years even, and they are doing it quite good, quite well. They are working, they are delivering and then it's really difficult to convince them to do it differently. How to do this. I would say that we work on all different levels. We start with the top management and the top management gives strategic direction. There's then also a bit of, you could say, pressure that you go in the right direction. We invest a lot in community. We have the testing guild, the community, we organize events, trainings. Upskilling is really, really important and we try to connect people because this creates a pull effect. When one product team where the people believe we are good and then they see another team and then they say wow, they make it different and they are much faster than we are. This creating a little bit of a competition between the teams and fostering this exchange. This is really, really important. But I have to say it's not an easy thing. It's a long journey because you often see the company starts DevOps transformation. They start and half a year later they are finished. What have they done? They have introduced some new tools, but with a new tool you are not agile, you are not DevOps, you just have another tool. It's not that we started yesterday or half a year ago with this journey. It already started two, three years ago. And it's a long gradual process to change the mindset.

Absolutely. And I guess being in a banking, it's more traditional. I'm just assuming, I'm thinking I used to work for healthcare, I used to work for Insurance, I assume it's kind of like banking. So a lot of pushback I used to get is like we have these 30 year old systems that they don't work with Agile or DevOps. Do you hear that Rudolph? And if so, how do you help change that mindset if you do get that kind of pushback?

When we take a look at this HR coaching competency model, you have different types, you have mentoring, you have teaching, you have technical mastery and so on. And yes, of course it is also the job of a coach to change their mindset. And.

We see teams which are working on the product since more than 15 years which are going into the direction of Agile and said yes, we need this. And then on the other hand there are sometimes where the people say, yeah, Agile, this new fancy thing where you do not need documentation and you are standing every day together for 15 minutes. So we're doing this since years, but we do not need this name. And then you need the right coach who can convince them that what is Agile all about? That this is more than only having every day a 15 minute stand up. And this is hard. And what we see in the new product teams which start with a new product, then we have these greenfield teams which starts with a product and there it is easy because most of these teams are fresh in their company, people who was hired with this Agile mindset and there it is, it is easy going to discuss being agile with them.

So implementing the model, it seems like you've done it for a year and a half now or two years since you initiated it. Any learning, like do you have like a before story and an after story of what maybe a team you worked with looked like before and then after the maturity model, after your coaches, after maybe what the results were, maybe David.

You can explain the team very broad, the external vendor being Agile, you know what I mean?

Yes, yes, yes.

I think this is a good approach.

Yes, here, this is a good example. This was one of the first reviews that we did and it was a product where we had an external vendor delivering software that had to be integrated and was also developed by the vendor still. And the outcome was really a lot of red, really, really not good. And we immediately defined measures and said okay, here we have to step into and work together to improve. And in the beginning it was really a little bit of a lost cause. Classic vendor, classic processes, own product development, not able to react to customer demands only after half a year or after a year. And we defined several measures together with this vendor. First we worked on the Deployment to automate the deployment step, that we have a pipeline beginning from the vendor when this vendor delivered some artifacts into our environments. Then secondly we created an acceptance test suite for both where we are developing test cases on the vendor side and on our side. And this is like the handshake in the pipeline then. And we also invested in the collaboration. We started to work in sprints. It's not yet that we deliver every two weeks. We don't have this cycle. But at least we are breaking down this half year or yearly releases now into two month releases where we have a couple of sprints. We still have a bit of waterfall ish elements in it. Like the last two weeks are acceptance testing and the business doing manual test. But we have moved. We came down from a release cycle from six months, two months. We worked together much better. And what was what the business, the product owner, the people who pay for the product really liked is that the quality improved the first time. Right? Because one of the issues was that vendor delivered the big release, a lot of defects popped up, missing requirements, requirements not really understood stuff that didn't work anymore. And it was the first time when we came into the cycle that really there was no major bug found in the end and they really could go into production. And this was one of the things that then convinced also the business that we are on the right track and it really brings value.

Maybe one input from my side, as David says. So the software was installed in our systems and then they immediately start with manual testing and doing manual testing for weeks and so on. And in the meanwhile they are installing the software. The installation now not only takes six weeks, it is faster. And then immediately we run our test automation and we verify with automated smoke test suite that the system is ready, that we can go forward with manual test. And the nice thing is this test suite is developed together with the vendor. This is not only implemented by us, this is a real collaboration where the vendor is creating, they care for the test automation and we take care for the test automation. And yeah, my heart is smiling.

That's awesome. So talking about your heart smiling in automation, you know, when I speak to people, I always assume that everyone has automation in place. How many of your teams had automation? Was it a struggle to get people to move some of their processes to an automation type pipeline or any learnings you had working with all these teams to maybe get them up to speed automation, you think people should have to get maybe a minimal amount of agile kind of transformation. Starting point.

We started on a very low level of test automation, it was really not very common. It was used more in UI oriented products, but everywhere else manual testing and discussed it even today with Rudyar. One of the big reasons for that is that we had this classic project approach and project management. And when you have a project that delivers at the end of the year after a long project, there is no need for test automation. And the project managers always said why it will be tested only once and some parts maybe twice. I don't need no automation for it. And now we have to switch to product development. There is a product that continuously operated, maintained and developed. You have a much higher release cycle and you go crazy with manual testing because you have to test continuously and this costs a lot of money. And I think this switch from this project approach to really devop and product thinking, this gave us a huge boost in test automation.

So two years before, before we started with this with the product teams we had so called capability basis and one capability base was software development and one of them was a software test. And if project needs testers then they go to the, to the capability base at to testers or to test automation engineers more or less not because most of the scene was in manual way. And after this project was done and the manual tester had did all this tester execution and so on, they go back into their capability base and move to the next project. And now due to these product teams, the product teams have to take care that the software works and therefore they recognize that test automation is a major thing in their sdlc.

Okay, Rudolf and David, before we go, is there one piece of actionable advice you can give to someone to help them with their transformations? Using the Agile engineering maturity model. And what's the best way to find or contact you all?

Yes, they care that you bring this mindset switch into the people and then they care for education, they care for upskilling and you cannot learn all these things by the way. So one important thing is upskilling. And this automation is a craft, you know. And you have to learn this absolutely from my side.

I have already emphasized it a couple of times. If you're using maturity tool like this, you shouldn't have only measurability and figures and management and all this stuff in mind. Yes, especially in big companies it's necessary, you need it, you need some kind of checking the progress. But this is only one side and the other side is this contact with the team, this communication, this continuous communication. And you have to earn the trust of the teams, you have to have role models to foster exchange. And this is really the hard part that the mindset change and the cultural change, but without it you will not be successful. You will get fake engineering and fake numbers and you think that you're doing something but in the end you do the same that what you did the last 1020 years.

Thanks again for your automation awesomeness. The links to everything of value we covered in this episode. Head on over to testguild.com a339 and while you're there, make sure to click on the Try for Free Today link under the Exclusive Sponsor section to learn all about Sauce Lab's awesome products and services. And if the show has helped you in any way, why not rate it and review it in itunes? Reviews really help in the rankings of the show and I read each and every one of them. So that's it for this episode of the Test Guild Automation Podcast. I'm Joe. My mission is to help you succeed with creating end to end full stack automation awesomeness. As always, test everything and keep the good. Cheers.

Thanks for listening to the Test Guild Automation Podcast. Head on over to testguild.com for full show notes, amazing blog articles and online testing conferences. Don't forget to subscribe to the Guild to continue your testing journey.