#037 - Lisa Crispin - holistic testing

Foreign.

Welcome to the no Nonsense Agile Podcast. I'm Shane Gibson.

And I'm Murray Robinson.

And I'm Lisa Crispin.

Hi Lisa. Welcome to the podcast.

It's great to be here. Quite an honor to be added to your list of illustrious guests.

Thank you. So the topic for today is Agile testing. Before we get into it, can you tell us a bit more about who you are and what your experience is?

I'm a tester by trade. I have a very long experience in the software business. Like 40 years and 30 years of that as a tester and the last 20, 20 plus years working as a hands on tester on cross functional agile teams. And I've been really lucky to work on a few really high performing teams and over the years collaborated a lot with Janet Gregory. We both started in the Agile world or the extreme programming world about the same time and we're looking for other testers to tell us what they do because nobody knew. And so she and I have collaborated on three books and we have a training company called Agile Testing Fellowship where we offer our courses around the world through training providers. I just recently got independent, so I'm not working as a full time employee on an Agile team anymore, which I miss a little bit. I live in Vermont on a little farm close to Canada with four donkeys, five cats, two dogs and a husband.

Okay, can you tell us what is the state of software testing today out there in the real world?

Practitest just did this state of testing survey. There's a lot more interest in testers working on cross functional teams and surprising amount of interest in testers getting more involved in continuous delivery and monitoring and observability testing and production. It's not that many people doing it, but it was more than I thought I would see. And, but also, sadly, it's still very siloed and very waterfall y and most people are stuck in very command and control environments. So that's sad. We've changed the name of our main course from Agile Testing to Holistic Testing Strategies for Agile Teams because everybody takes for granted they're doing agile whether they're really doing it or not. And it just doesn't mean a lot now to say agile or Agile testing. But for us, holistic testing means something very specific. So that's why we're changing our terminology.

Yeah. So can you tell us more about what holistic testing is?

Oh, I should pull up the holistic testing model on the agiletestingfellow.com blog. The second blog post down is Holistic testing what it means for agile teams. So it's just a way to think about testing through that whole development lifecycle. It really bothered us when people start talking about shifting left and shifting right. Software development is not a linear process, so what could that possibly mean? So let's go with holistic testing. The whole team is responsible for quality. The whole team should be responsible for making sure testing activities get planned and done at the right time. And the whole team is just going to participate in this all the way around the cycle.

One thing you've got in there is this idea of testing as part of the discovery phase. I really like that because I don't see that a lot. I see a lot of people talk about testing as testing the software, testing the release, regression testing, very technical testing, but not testing our hypothesis.

One of the biggest differences between agile testing and traditional waterfall type testing is in waterfall you're focused on defect detection and on Agile we're focused on bug prevention. We want to build the quality and we want to prevent bugs as we're writing the codes.

It's interesting because we often don't see people with testing capabilities come into that UX design process. Testers tend to be negative. They tend to look for the things that are going to go wrong because that's what their skills are.

In my experience, we testers tend to be keeping the big picture in mind. When you're writing code, you have to be very focused on what you're doing. It's a very narrow focus and you have to be positive about it or you would just get discouraged and give up. We're more big picture people. When some product person says, oh, this is my idea for a feature, we start talking about it. I might first say, why are we doing this feature? What's the purpose? But one of my early questions will be, what's the worst thing that could happen when we implement this feature? Elizabeth Henriksen has a great game in her Explore it book called Nightmare Headlines. You know what don't we want to see on social media the day after we release this feature? That is a negative way of looking at it, but it's also a way to help with lateral thinking. We're all such slaves to our unconscious bias. And so if we can ask some questions to get ourselves thinking out of the box, that helps. Pete Whalen years ago, his business card said QA question asker. And I really think that's one of the things we're good at. That's why we're such a help on the right side of the DevOps loop because we're looking at log data, we're looking at dashboards, we're spotting anomalies, we're spotting patterns, we're identifying risks, because we're asking questions of that production data. And we can dig into it now that we have all this big data and all these tools to explore it and analyze it. I think that's one reason that I really want to get more testers working on that side of things as well. Don't just throw it out the door and start working on the next thing.

So you just said quality assurance, and there are places where quality management and testing are considered to be totally different things done in different silos. So quality management is all about processes and compliance, and testing is about bugs and defects. What is the difference? Should they be the same or separate? Or what are testers, QAs or not?

QA has always stood for quality assurance. We cannot assure quality. You can't test quality into a product. It's already there. I hear people call testers QAs. It's like the initials don't even mean anything anymore. I think a lot of people have moved to quality engineering as a better way to describe it. They're welcome to that term. Some people in the testing profession feel like tester is not a prestigious enough title.

Yeah, but should QAs get involved in defining quality control processes, quality management processes, as well as testing?

I think they could, but I think it's a different field. I am involved in quality processes, but just from a software development perspective.

So let's say that you are starting with a more traditional testing team. They're working in their functional division. They start writing their test cases while the developers are coding, and they don't test anything until there's been this big release into test.

Don't forget, Murray, we gave them three months to test, but then we were a little bit late in our development processes. So they've got one day to do all their tests before we go live.

And then if it all falls over in testing, obviously the tester's fault.

Oh, and by the way, as well as being late, the developers forgot to document anything.

And the requirements have all changed as well. So all those test cases you wrote, you have to rewrite them all now.

But we didn't update the requirement documents that testers had for the last six months. But we haven't to tell them that things might have changed.

And fortunately, though, we didn't automate anything, so we don't have to rewrite any automated test cases. So thank God for that.

Well, that's not even a good waterfall team. So I've had the privilege of working on a really excellent team or organization using waterfall as a process back in the early 90s. But guess what? Representatives from test development and from operations participated in all the analysis and requirements. They didn't do test driven development, but they did in the 90s percent coverage with unit level tests. We tested at the UI level, we automated all those regression tests, we had continuous integration, we had automated deployment to 20 different UNIX environments. We only release every six months to a year because we were a database product. And that's all you needed to do back in the early 90s. But good practices are good practices. I don't care if you're a waterfall or you're agile or you're whatever.

I worked as a project manager in a waterfall environment with quite well developed processes. And the idea was that you would recruit your test manager after you'd done your architecture to start writing the test strategy in the test plan, because obviously they needed all of the requirements and the architecture to do that. And testers wouldn't come on until development began and you'd have some of your testers come on to write your test cases and then the rest would come on in India or somewhere to actually do the testing later. So that was considered to be standard because it kept the cost down and all the documents were produced. How do you transition from that functional test team that does things afterwards to a agile team?

A lot of the time people hear the word agile and sprint and think we're going to go faster now. And the truth is you need time for the team to learn how to self organize. You need time for the teams to decide what technical practices they're going to use and then learn those technical practices. I think one of the reasons test driven development has not been adopted widely, even though it's been shown to prevent 80% of the bugs that are otherwise found later, is that it took about eight months to get any traction. Brian Merrick called it the hump of pain. You're learning to write unit tests and do driven development. It's more work. You are going to go more slowly, eventually you're going to get more experience. You build up a library of test components and you get over that hump of pain. And now you're saving effort, now you're going faster, you're going to go save effort, which now you can use to build new features. I've been lucky to be on teams for enough years to see the magic happen.

So let's say I'm doing this because the team hasn't been producing much and we need to get more effective and efficient and faster and better quality.

I would have a retrospective and I would say, okay, so let's put all our problems up on our virtual whiteboard, on our virtual sticky notes. And now let's dot vote and figure out what's our biggest problem. And now let's design an experiment with a hypothesis that we believe that if we do X, then it will result in Y. And we'll know by this measurement within some short period of time. Small, frugal experiments, as Linda Rising Sundays. So within two weeks or four weeks, we can know if it's working toward the goal that we want it to achieve. And if it's not, we learn something from that experiment and we can tweak it or try a new experiment. We can't fix everything at once. We can only fix our biggest problem at a time. We're just humans. We can only do so much. I would encourage people to build relationships within the team and to other teams, especially the infrastructure type teams. And I would advocate working together in pairs or even better in ensembles. Cross functionally because that saves so much time. When you have a tester in there, you're going to find the bugs during coding and that later if you have a designer in there, you're going to get the design right right away.

So you're talking about a code test pair. So the developer is starting the code and the tester is working with them on the same code at the same time.

Exactly. One of the mistakes I see newbie agile teams make over and over is they want to make the business people happy. So, so they agree to take on all these stories one iteration and they can't finish any of them. And so one of the things that I try to help people do is commit to less. I know we don't commit anymore in Scrum, but people still think you should finishing one thing at a time. So at the end of this branch or iteration, we have something we can deploy, maybe even something we could release. So one thing at a time, baby steps.

Okay. Should we even have people with a test role in an agile team?

Depends on the team. I think most teams need somebody who's a professional tester and has that expertise. I do think that depending on the rest of the team and how open they are to taking responsibility for testing and quality, that tester is more of a test consultant, helping the non testers learn the skills that they need. So for example, in my Last full time job, I would either sit in an ensemble with other developers or maybe there's two developers pairing and I'm in with them. I'm not going to take a turn driving, I'm not going to write code. But as they're working, I'll say, I think it's supposed to behave this way. Yeah. Oh, okay. Or I'll say, I noticed this code is duplicate of what you had over there. Could that say something that should be abstracted out into a helper or something? Oh yeah, you're right about that. Or just raise an issue and it's like, you know what, we better go ask the product owner what they want or we better go ask the designer. We better get the designer in this zoom call and find out. They get so heads down coding and they don't necessarily stop and sit back and say, are we actually building the thing that was desired? And I would ask for feedback. Am I slowing you down asking all these questions? And they're like, no, no, because it's going to save us a lot of time. If we thought we were done and then they threw it back to us, that would be a real waste of time.

Do you see mature teams still having testing as a separate thing in estimation that they focus on rather than it just being embedded in what they do?

I'm not a fan of estimation. I've found it much more productive to have the conversations, but slice all the stories down to where they're the same size. Like they're all one point stories and we just count stories. It's getting the stories right size that really matters. If you're doing Scrum, you can write test cards for that story. And if you're not doing Scrum, it's going to be a working agreement. We agree that we're going to write unit tests because we're maybe we're doing test room development. But even if we're not, we're going to write unit level tests. We're going to automate tests at the API and UI level for this story, if that's appropriate. We're going to do the exploratory testing we can do at the story level. And that's all part of the story. And so we size that story accordingly. And if testing is really hard, testing might take longer than coding. We have to be aware of that. But it's also important to do other types of testing that can't be done at a story level. So what's worked really well for my teams is writing exploratory testing stories in the backlog along with the feature stories. And when enough stories are done that we can test a workflow end to end, like a real user uses it. Then the team starts pulling in the exploratory testing stories and doing those. And again, it's not up to the tester necessarily do those. Usually the tester is going to be involved in writing them. Although I have worked on teams where we held workshops and paired with the developers to where they all got pretty good at exploratory testing. You might also have stories for testing infrastructure. If you're doing infrastructure changes, you might have stories for testing, the monitoring, the logging, are we capturing the right information? You might have stories for doing accessibility testing because you need to do more at the big UI level. So security testing might have stories for that. I like to have them all visible in the backlog. It's all work we have to do. We're going to have risk at releasing that product until those stories are done.

I had been having a conversation with people recently about how to do system testing. When you haven't built the system yet, let's say there's some part of a system and you're expanding it. How do you set up this automated continuous delivery environment if you haven't built all the interfaces and all the different components yet?

I feel like that is where it's useful to have a virtual quality team, where you take people who are testing professionals on these various feature teams and maybe infrastructure teams, and part of their job is not only helping their team, but getting together and saying, how can we judge quality at the system level? What data can we use to know and what kind of extra testing might we need to do that? I'm a fan of building things and to end doing something like story mapping, where you lay out your scenarios and then you choose something that you can get working that's valuable, or the most important thing for your customers and the most important thing for you or your business. Get it working, get it in production, and then you're going to have all the underlying infrastructure to support that. So that all has to be done together, but it doesn't have to do very much. Maybe it's not even useful to your customers at first, but it's useful to you because you can automate tests for it, you can make sure it's working. So doing things slice by slice.

I've asked people to set up end to end system testing for the current state so that as we change the system, we can see what happens. Quite often people will argue we don't need to do system testing because we're just making a local change and that won't impact anything. I say fine, then run your automated system test over it every night, just to make sure, because you don't know what all the consequences are.

Now that we're dealing more with microservices and with applications that are supposed to have stable interfaces, it feels like we should be doing more API integration testing than we used to, instead of focusing so much on unit testing. What do you think about that?

That's a big trend in testing. So there's a lot more adoption of testing at the API level and also doing contract testing, which is not just testing the contract between those services, but that's important to do. And having that just as an isolated thing, I think is really productive. Mark Winteringham is about to publish a book on API testing, which I meant to be reading. I love testing APIs. People have realized you can test a whole lot of the business functionality through the API level. There's a lot of interest in API. There are a lot of great tools like Postman and Jest lets you do a whole lot of functional testing in the ui. So we have a lot more technology at our disposal and the more we can isolate that, I think it's really good. Test Automation University has some good courses on various types of API testing. So it's really exciting to see people really embracing that.

So can you explain the test Pyramid and how this change relates to that?

The idea of the Test Automation Pyramid is push all the tests down to the lowest level that you can. At the bottom, there's unit tests that are fast to rate and give us that fast feedback and they're easy to maintain. Those are the tests we love. And then that middle layer is, yes, API level, service level, they're a little harder to rate, they're a little slower feedback, but we don't hate them, really. They're okay. And then that little triangle at the top are those UI level tests that are actually workflow tests, not UI unit level tests. But those are the ones we hate because they get flaky and they fail. And that's the way to look at it, is push them down as low as you can because those are the tests you love and so that API levels in the middle and I can really see these days, for a number of reasons, your shape might be different. So we have a lot of really cool UI test tools and then we have a lot more we can do at the API level. The Test Automation pyramid is just a thinking tool. You don't have to end up having a pyramid shape.

Could you do me a favor and describe tdd, ATDD and BDD and what the difference between the three of them are? Because I constantly get confused when I'm talking about them.

TDD is test driven development or test first development. And that is actually a co design practice. It's not actually a testing practice, but you end up with a nice suite of regression tests that provide a nice safety net for you. But it's right a little test for some little tiny chunk of functionality. Write the code to make that test pass and then move on to the next little chunk. And you're going to be refactoring in there so it's red green refactor acceptance. Test driven development is okay, we've got this feature. We're going to try to build some shared understanding about it. We're going to slice it into stories. We're going to talk about the story, make sure we understand the story. Do something like example mapping to get the purpose, the business rules, concrete examples. We're going to turn those concrete examples into executable tests that guide development. And so the developers are going to start with the happy path test, get that passing. Then they're going to move on to different tests for boundary conditions or edge cases or whatever, get all those things passing. We're going to have conversations all along the way as the product owner. So we're going to find bugs, we're going to fix them right away. Behavior driven development is very similar to accepted test driven development, but you can use given this when I do that syntax so it is English and business people can follow it. And so BDD is making sure that those tasks are readable by the business people, product people, so that they can participate in creating them in pre iteration planning meetings where we talk about the stories. I like example mapping from Matt Wynn, but there are other techniques to have a structured to make sure that we're doing lateral thinking about this story, is it the right size? So having those conversations before we do the iteration planning meeting so that we can have that foundation of shared understanding. But these days BDD is used more at the acceptance test level or story test level. And then there's also specification by example from Goyco Adzic, which is a very similar type technique. So they all have the same goal, building the shared understanding, writing the executable tests and using those executable tests to guide development. So we know what code to write and we know when we're done writing it.

So Found that natural language that given blah blah blah and blah blah blah, then aligns with the way I think more than code. And so it feels more natural a way of writing tests or describing how something should work. What we found in the data world is we can actually push that earlier in the cycle and use it to define requirements. So we can say, okay, we need to go and create a flag which is active marketing customer. And the rule will be something along the lines, given there's a customer and they have an account and the account has had a balance in the last three months and the person's not dead, then they're an active marketing customer. So do you find that that natural language, behavior driven design pattern is getting pushed into the specification side of things or is it still being treated as a way of describing a testing?

I think what you're describing, they're examples and they're awesome. And there's something that we need to do together. We need to have a product person, we need to have a coding person and a testing person sitting down together to write those. The unit level tests are all about code correctness and internal code quality. So that's owned by the team. The tests you're describing are business facing. Those are really the quality as defined by the business and hopefully that matches the quality as defined by the customer, which is really the only quality definition that counts. If you never automated those tests, I wouldn't care so much because the value is in writing them together and building that shared understanding. And it's funny because when I've worked on teams doing that, I couldn't tell you how many times it happened that I wrote a ATDD type of test and the developer comes over and says, oh, this test is failing. And I say, well, that's because the it should be doing this. And you wrote it to do that. And he said, no, I think it really should do that. Now we got to go to the product person and get the answer. And it makes us talk to each other. That's the value. And we did automate a lot of. We didn't keep all the ones we used to guide development because it was overkill. You can't just have these gigantic suites of automated regression tests running even at the API level. So we kept the ones that made sense, but doing all the detailed ones made sure we all had that shared understanding. We built the right thing. Less rework, less waste, shorter cycle times. That's what we love.

Yeah, less waste, shorter cycle times. All that sort of stuff is great. Should we be aiming for Zero defects.

Personally? Absolutely. I've known teams who've succeeded at that. I've been on teams who pretty much succeeded with that to where the point we didn't really need a a defect tracking system anymore. So it is an achievable goal. And if you're doing ATDD or bdd, you're going to probably catch defects in development. It's doable when teams are really competent in writing really solid, robust code that's not going to fall over. The bugs are misunderstood requirements. And if we focus our energy on getting those right to begin with, but on the left side of that loop, we're not going to have those problems.

What do you think about the idea that quality is free? That all of this time and effort we invest into testing has such a great return on investment that it more than pays for itself?

Testing is part of software development just like coding is. Just like design is, just like architecture is. It's not a separate activity. It's not something we should split out. Do we put a cost on coding and look for ways to save money on coding? Maybe people do. I haven't ever heard of that conversation. So it's an integral part of software development and it needs to be done right along with coding and quality. Again, that's up to the customer. Do we want our business to do well? We need to agree on what level of quality we want to deliver so that our business can do well because our customers are happy. Teams really need to have that conversation. What level of quality do we want? What can we commit to? It needs to be a commitment because you're going to run into obstacles and if you're not really committed to delivering that level of quality, you're just going to give up. Then our customers are going to suffer because our code's not going to be very good quality.

I've seen people talk about testing quadrants, but I don't really understand it. What is that?

The testing quadrants are another thinking tool that helps us think about quality or testing from technology facing side. What do we as a technology team care about to make sure that we're happy with our internal code and framework correctness? And then business facing quality. What does our business want? What do our customers want? And then what helps us guide development? And then what helps us evaluate what we've built after we've built it? And then the stuff on the right side of those quadrants feeds back to the left, just like the DevOps loop. And just a think of tools. It's really handy when you're planning a new product, a new feature, even a story of what all the kinds of testing we're going to need to do for this performance testing is a great example. A lot of web applications and web products live and die by their performance. But people leave it to the end. We're going to do that right away. We're not going to leave it to the end. The quadrants imply no order. This is the quadrant four. But we're going to do it first. Same with security testing. Let's make sure our framework is secure, whatever it is. So it's just to help us think laterally, getting over our cognitive biases so that we don't get bitten later by the unknown unknowns that we didn't think about.

I have one last question for you before we go to summaries. What's a really good agile team look like from a tester's perspective? What's the result? What does it feel like? Can you describe it?

It's interesting that you ask that because Janet and I talk about this sometimes. If you have not experienced the unicorn magic of being on a high performing cross functional team, it's not something you can explain to somebody else. You have to just feel it. For me, it's a team that has had that conversation about quality and really cares and really has committed. They've decided I want to wr code that I would be proud to take home and show my mom and put on my refrigerator. And I'm committed to making that happen. No matter what crazy obstacles we run into, whether it's our management or our organizational structure or the architecture team, we're going to find a way. And being brave enough to learn the good practices that might be difficult that we know will give us good results. Now of course that assumes that we have management that lets us do it the best way we know how. Just teams where every competency is equally valued and every voice can be heard. Psychological safety obviously prerequisites, as Joshua Karaefsky's modern principles say, everybody can raise issues, everybody can ask questions. Just that safe environment and the we love our customers and we are so happy we can make them happy.

Great. All right, let's go to summaries then. Shane, what you got?

All righty. So you talked about waterfall, where we focused on defect detection and we talk about agile, we focus on bug prevention. But then quite rightly, you took Murray and I to task when we described a bunch of very bad practices and called them waterfall. So I'm going to go and say that bad process is where we look at deep detection at the end of the process and a good process is where we look at bug prevention by doing things early to stop them appearing. I love the idea of having people that have testing skills in the UX process and that's because they're the question askers. They tend to be good at listening, understanding and then asking the questions. They have that mindset. I'm assuming that's because they have a set of either conscious or unconscious patterns in their heads where they're looking for those things are going to bite them in the bum. So I like that. One of the things you do is ask the team what's the worst thing that could happen and that will bring out some of the highest risks. I think that idea that there are a bunch of good practices and processes and patterns that teams can adopt, things we can use to help our thinking. So the testing quadrants, the test pyramid example, mapping a bunch of things that actually we can help to improve our skills and our capabilities in that space. But we should always start off with test coach. If we haven't done it before, we don't focus on the amount of change we're bringing to teams on day one when we help them adopt this new way of working. I hadn't thought about testing as yet another thing that we encourage them to relearn and that's going to take time. I like the idea as well that describing a test helps gain a shared understanding. By actually describing what a test is, the different people that are working on that piece of work will have a better conversation. They should have a better shared understanding. And the last one for me, which is definitely my takeaway, do work that you'd be proud to take home and put on your mum's fridge. So there's giving my two mantras, agile ways of working. They're successful. When teams are having more fun and they're taking work home and putting on their mum's fruits that they're proud of. So that was me, Murray.

I'm glad you said it, what you've said, because this is what I tell the teams I'm working with. I don't know as much as you, but I do tell them to do all of those sorts of things, but this is something we've got to work towards. It can take quite a bit of time. I think putting test specialists and dev specialists together in one team is a really good start because that brings testing to the front and then having the test specialists work with the product owner or the requirements people to flesh out the requirements. I very often suggest people do that. I haven't thought of pairing developers and testers before. I'm going to start suggesting that to some people. I think it's a very good way for them to understand each other. I think an agile coach should be talking about the stuff you're talking about. Lisa, I don't like the idea of agile coaches who don't know much about testing or software development. And there are some of them. I think agile coaches should really learn as much as they can and read your book on agile testing and more agile testing so you know what they're talking about. I think it's really important. I have worked on a couple of great teams where dev and test have worked very well together. And we actually developed a lot of our test practices through retrospectives. For example, before the developer puts a ticket to the tester, why don't they have a conversation? And the developer says to the test specialist, I've done this and I've had a code review and I'd like somebody to test it for me. What do you think? And the tester can say, well, show me. And they can just say, look, I can see you missed this entire requirement, so it's not ready. And then you don't have to go through the whole test phase. That's something came out of one of our retrospectives I had with a really good team. It's a beautiful thing when developers and testers and everybody else are working well together and the quality is high. It's amazing how efficient you get when you're not spending all your time dealing with defects and trying to fix them. People find it hard to believe. I remember arguing with a client about a contract once because he was insisting that they wouldn't do any user acceptance testing until the release had been completed and then they wanted to raise defects. And we argued about whether the contract should allow them to raise defects on defects. And I said it's a terrible idea to leave all your user acceptance testing until we have a release. I want you in there right at the beginning with our test team and our dev team helping us to make sure that we're finding problems and fixing them straight away. And I'm worried that if we have this contract, then you're not going to get involved until the end and then we'll have all of these issues propagate because we misunderstood things. So I love the idea of bringing the customer or the client or the stakeholders in early as well.

I'm glad you mentioned that. Show me. I think when you're talking about how do you get these people learning to work together that never worked together before on a new agile team? That practice of ask the developer. Oh, can you show me what you've just coded before you check it in? I just like to see how it works. Let's look at it a little bit together. It's just going to take a few minutes, but it's going to start building that relationship and people will start seeing the value of working together.

Yeah. All right, thank you very much for coming on. Now, how can people find you?

I'm very unimaginative. I'm just Lisa Crispin on Twitter and LinkedIn and my website is lisacrispan.com and.

You'Re available to help people with what sort of consulting are you doing at the moment?

Janet Gregory and I have our Agile Testing Fellowship where we offer our holistic testing course. Holistic Testing Strategies for Agile Teams. We're also working on some new courses including Holistic Testing for Continuous Delivery is going to be coming up. So I could facilitate the training, which is a lot of hands on, really fun stuff. And then I also can help teams with things like quality practices, assessments, do process retrospectives and interview people and find out how are you doing quality now, how are you doing your testing now, where could you improve? And then I can maybe offer some workshops or one on one coaching or something like that to help improve transformations. Teams that are trying to fit testing into continuous delivery. I still see teams that have embedded testers that are calling themselves Agile and now they're going to do continuous delivery and deploy every day. And why isn't that tester keeping up so, yeah, those kind of things. That's one of my favorite areas to help people with. And then our book website is Agile Tester.

And you mentioned our books Agile Testing and More Agile Testing. But if you haven't read those books, the place to start is Agile Testing Condensed, which is our latest book available on LeanPub in many languages and on Amazon as a print copy because it's only 100 pages long and it has some information to see the other books plus some new information, but it has a lot of links. So if you find a topic that, oh, I want to learn more about that, then you can go at one of the other books that delves into it more deeply. And all of those books we gathered stories and experiences from leading practitioners all around the world because I think we all learn really well from hearing other people's stories. What problem did they have? How do they solve it? And So I think that's the most powerful thing in our books and the thing we get the most feedback on, what was most helpful. Thank you very much.

Thanks for coming on.

Thank you. It's a great honor and wonderful to talk to y'. All.

Yeah, it was nice to finally meet you. And again, just a shout out for all the testing stuff that you share on Twitter and LinkedIn around the world. I've learned so much by just reading it from this end of the world.

Great. That was the no Nonsense Agile podcast from Murray Robinson and Shane Gibson. If you'd like help with agile, contact murrayvolve.co that's evolve with zero. Thanks for listening.

Sam.