BDD in Action with John Smart

Get ready to discover the most actionable end to end automation advice for some of the smartest testers on the planet. Hey, I'm Joe Calantonio, host of the Test Guild Automation Podcast and my goal is to help you succeed with creating automation awesomeness.

Hey, it's Joe and welcome to another episode of the Test Guild Automation Podcast. And you're in for a special treat. I cannot believe I have not had John on the podcast before. So today we'll be talking with John Ferguson Smart all about BDD in action, the go to book. I always recommend anyone having to do anything with BDD or even automation because it dovetails bolt so perfectly. So if you don't know John Ferguson, Spot is a specialist in BDD automation testing and software lifecycle development optimization. He is the founder of the Serenity Dojo which is an awesome online training platform for testers who really want world class agile test automation engineers how to become one that shows you how to do that. And he is the creator of the BDD or the Serenity BDD Test Automation Framework. You don't want to miss this episode. Check it out.

This episode of the Test Guild Automation Podcast is sponsored by the Test Guild. Test Guild offers amazing partnership plans that cater to your brand awareness and lead generation and thought leadership goals. Get your products and services in front of your ideal target audience. Our satisfied clients rave about the results they've seen from partnering with us, from boosted event attendance to impressive roi. Visit our website and let's talk about how Test Guild can take your brand to the next level. Head on over to test guild.info and let's talk.

Hey John, welcome to the Guild.

Hey, thanks for having me. I'm glad to be here.

Great to have you. Really excited.

It is surprising I haven't done this before.

Yeah, yeah. I can't believe like it's over nine years, still haven't had you. So today's the day. Really excited, John. I always botch bio. So is there anything I missed in your bio that you want the Guild to know more about?

You covered a lot of stuff. You covered a lot of material. But that's basically what I do. I help teams communicate better and do better test automation. I try and run the Serenity Dojo which helps testers become top world class test automation engineers. We use education, science and all sorts of fairly pretty cool stuff to. We've come up with a program which I think is pretty original. I don't know anyone else who who really teaches that way and it does sort of work really well, why is that?

What is education science then? Or is that your secret sauce that you can't reveal?

It's a secret sauce, but basically what I found was that I was training testers for, I've been training testers for, for quite a while and I was using the usual sort of boot camp standard course type approaches and especially in corporate context, but even with individuals I found that that sort of very structured approach never really, I mean it feels good but it doesn't sink in a lot of people sort of afterwards it they go back to their old habits or they go back to their old ways or they don't really adopt the learning that I was trying to commit. So I actually had a conversation one time when I was in Sydney with a friend of mine from high school in fact, who is now very knowledgeable in education science and he showed me some techniques which basically throw all of the conventional boot camp training approaches away and use a much more intuitive, natural way of learning. Flips everything around and it's really, really effective and sort of. I started doing that in a corporate context in companies that I was working for and then started using it for individuals. And yeah, it works really well. So I'm really pleased with the outcomes. Again, it gets people, they're getting promotions, they're getting new jobs, they're getting more job satisfaction and it's really good to see just the way they, when you see them posting things, even posting things on LinkedIn or talking about what they're doing, actually practicing the practices that we sort of often talk about and doing things properly and helping other people as well. So it's really satisfying.

That's awesome. You know, I get asked all the time, hey, are there any good boot camps I should go to or I don't know anything about automation, what should I do? So is the Serenity Dojo appropriate to all types of levels or is it. Do you need some sort of proficiency in automation before entering the dojo?

Because it's a tailored approach, it works for all levels. We've got people coming from purely non technical manual backgrounds with no coding skills at all and we bring them up to speed in Java and JavaScript and we're doing Python soon. Whatever work, whatever is the most appropriate tool for them and for more experienced testers or even we've got some managers or Scrum masters or sort of QA managers who come on the program as well, who want to understand the, what the automation is all about, what the automation their teams are supposed to be doing and to be able to Advise them and direct them a bit better. So it really does cater for all levels.

Great. And John, I assume you're still a believer in behavior driven development. This is one of those topics that drive me mad, especially online, where I think people really don't understand it and especially when automation gets involved. So could you just tell me a little bit more, you know, what is behavior driven development? How did you get involved with it? Why do you seem to be such a great proponent of it?

For me, behavior driven development is basically agile, done well with a whole lot of practices around getting collaboration and communication, actually making it effective, basically. So there is the side of. So traditionally we say BDD evolved from TDD because it came from Dan north teaching TDD and finding, figuring out that if you didn't actually talk about tests, you could teach the concept a lot better. But since then, that was in the mid 2000s, the noughties. Since then it's evolved a lot and become very much a team level collaboration practice. I mean, there is a strong association in people's minds with tools like Cucumber and Gherkin. But really if tomorrow we get a better tool than Gherkin and find some other way of automating our specifications and we're still doing bdd, it's really a collaboration practice, a flow, a way of getting teams to articulate their requirements and agree on their requirements and then express those requirements in a way that flows naturally into automated test. And that gives, for one thing, that makes it much easier to do automation because you can streamline the automation process and actually do instrument automation a lot more easily because effectively you're doing. There's a whole lot of acronyms, but you're basically doing test driven development at a domain level. You're designing the examples and the, that you want to show to your product owner or your users or your stakeholders at the end of a release or at the end of a sprint. What are the examples you want to show that will provide evidence that you've actually delivered something? Let's automate them in a way that they will understand and everything starts from there. So you flip things around. You say, right, what are we going to demonstrate? Let's build the automation to show that, to demonstrate that. And then sure, you can do other automation under the hood and you'll often do TDD at a more granular level. But so what I call this table stakes. You get the, that's the minimum automation that you should be doing and then that frees you up for exploratory testing and Other stuff as well.

Absolutely. So you know mentioned communication is key. I love that BDD is agile done well. That's a great quote. So do you find teams struggling with communication and that's why somehow their bdds or automation gets kind of wonky at some point because they kind of miss out on that. Keep point is communication.

If you get clarity on requirements across the whole team, you get a whole lot of defects that just don't exist, that don't happen that would normally have happened. And conversely, when you work in a siloed way, which is what happens when we don't get the communication work, even if we're doing scrum, if we've got developers working separately from testers, testers pick up the pieces and then test you always, you get that disconnect and you get that trying to play catch up or trying to the whole unplanned rework which we're in the lean world in Kanban that we're trying to avoid. That's where it all comes from. That misinterpretation or misunderstanding or lack of clarity around what the requirements are or if the requirements come into a sprint, come into a team and then the product owner might not have clarity on them, but the product owner I've had, I remember that's a really interesting animated conversation stand up a few months back where the product owner was adamant that the requirements were clear, but the developers were not so adamant. So their definition of clear requirements or what was the word they were using, it was a word that I felt like saying, the Princess Bride quote. But you do get. When you do BDD properly, the whole point of that is to take a requirement, say, right, well, this is what we think it's supposed to do. Have we understood this correctly? What are the assumptions? Let's walk through this and make sure we're all on the same page. And let's not just do it as a random conversation, but do it in a structured way so we can come up with a few key examples and counter examples. And. And it does take more effort initially because it forces you to sit down and think. But that pays off because you've got that clarity about what you want to build. And that's where you get like teams doing BDD well, they get 80, 90% fewer defects, they don't get production incidents and get deliver much faster or get much. Even though they're spending more time thinking about what they have to do or talking about what they have to do initially, they end up delivering a lot faster because there's not that back and forth and uncertainty and rework that happens. I mean it's again these whole concept of agile collaboration face to face communication, but putting in place a whole lot of structure to make that happen and to make that effective.

Absolutely. And you mentioned structure. So I think that's another kind of hurdle for people is not only communication, but then how do I incorporate BDD with Selenium or whatever automation I'm using in the background? And that's where I think Serenity comes in. Serenity gives people almost like the scaffolding for everything they need for BDD in a way that they kind of not makes it easier, but kind of facilitates how to do BDD correctly. So I guess first, can you talk a little bit about Serenity? Gosh, I guess it's been around for eight, nine years now. It's the framework I used when I was actually working as a automation engineer at GE way, way back. So a little bit more about Serenity and how maybe it can help make BDD easier for people.

Yeah, well, Serenity actually predates, well, it doesn't predate bdd but the first version of Serenity were BDD without cucumber. That was just pure Junit. But the idea was, has always been that it's a living tool. So how can you write tests, how can you write automation in a way that actually communicates to the business and provides evidence of what you've actually tested? And now it works with obviously with cucumber also we can. There's Serenity JS that my colleague and friend Jan Molech's written where you can use TypeScript so the whole and integrate with any of the software node JS tools and get the same sort of outcomes. So the big thing with Serenity is that it allow a lot of people often ask, well so I understand that the BDD scenarios are meant to not commit to the implementation too much, not talk about clicking here and clicking there and not being full of UI stuff. But how do we provide evidence to users or our customers or our stakeholders that of what's actually being tested? I mean I work a lot in banks and they're all about they have to have evidence of what was released. And so that's what Serenity does. It documents the steps in business terms from the cucumber level, not as or even if it's with Junit or Mocha or whatever. From the test automation level. It documents each layer going down right down to the API calls or the clicks or the database queries or whatever. However you're interacting with the system at each step, it explains what that was trying to achieve. So it's very much built around the whole concept of where tests should not model the application, they should model the behavior of the user that you're trying to simulate. And so you want your test to map out the user journey and document that user journey, both from a high level and at a very granular level. So that's what Serenity is all about, being able to provide that evidence and that documentation. And it does. Along the way it adds a whole lot of the moment's mostly Selenium, except in the Serenity JS which does Playwright and I think Cypress as well. Now I think is integrated with cypress. Possibly not 100% sure on that one, but webdriver IO I think as well on the Java side there is a POC for integrating with Playwright. I've just got to get around to finishing it, but that's on the way because Playwrights does work in Java as well. But it works with rest assured, obviously. So it's not real. It integrates with other testing tools to provide that layer of living documentation and that structure around the test and that reporting that otherwise you'd have to set up yourself.

All right, so that's another piece I think people don't get is that really drew me to Serenity is the documentation piece. And when done in the way you mention is you get living documentation not only for your users, but if you have a new tester come on board or something, if you do it correctly, they could read the output of your test and learn the system under test almost. So it's like it has multi benefits when done correctly. And when you work for like a healthcare system like I did, if you get audited, rather than have them look at the code, you just show them the output of your test. And it should be, you know, it'd be written on a way that they could follow along to see what was tested and what was verified. So is that correct?

Exactly.

Why one of the main benefits?

Yeah, absolutely. Yeah. Again, like I said, I work in banks. There's a lot of compliance questions, a lot of how do you actually document that? Other than creating a Word document with lots of screenshots, we can also do it automatically.

So this brings in, I think the main event here and that is BDD in action. I have the first edition, as you can see, it's highly well for the people that watch on YouTube, not the people that are listening. I have a joy gears all over the place for this book. It saved my life when I was working like I said full time as an automation engineer. So John, to me this book was so complete and like I recommended to everyone. Why a second edition? How could you expand on and make better what I thought was like one of the. What I would recommend as the go to book for anything to do at BDD or automation.

Well it was, it was. Came out in 2020 13, 2014.

Yep.

A lot of techniques have emerged in sort of the BDD world since then. So at the time we were. The techniques that we used have evolved the time we would write given when then scenarios directly in Three Amigos now we tend to do things like example mapping or feature mapping or other techniques, other brainstorming techniques. We use Figma or Miro online to come up to work around the ideas and brainstorm the examples and counterexamples. So there's a whole lot of collaboration techniques that have emerged that I thought it was important to document that weren't around just weren't around when the first edition came out. So that was the, the biggest thing the. It wasn't intended to take so long to. I think I came up with the idea in about 20, maybe 17 or 18 to do a second edition. So it took a little bit longer than I thought it would. It was one of those projects we do this not because it's the right thing because, but because we thought it would be easy.

So what else was added then? Why did it take so long then? Was it just because life happens and.

It'S because we basically re updated every single chapter and also I got Jan Mollach to come in and do some stuff on Typescript and Serenity js. We did several chapters on the screenplay pattern which has emerged since. Well, which was around but it's really sort of become a bit more mainstream since then. We've introduced with example mapping, journey mapping, feature mapping, all of these collaboration practices. We talk a little bit more about microservices and microservice architectures. So every chapter pretty much has been and all of the code examples obviously have been updated and with the latest versions. So it's pretty much total overhaul. The title stayed the same.

Nice. So like you said when this book came out, Serenity js I don't think was. Was even in existence yet. So is it the same tool or is it basically like how does someone choose then? I guess if they're using Java they use Serenity. If they want to use JS they use just Serenity js Do they have exactly in Parallel with the features and such, other than what you mentioned is one has may support for WebDriver IE and the other one has Playwright where Serenity is still not doesn't have that support as of yet.

So the main differences are Serenity JS is a pure Screenplay implementation. Serenity has Screenplay plus the Action Classes model, which is basically classes that you use to document your steps, your steps and methods and they get documented in the reports. And then inside the steps you can use more traditional page objects. Whereas Screenplay is Screenplay in Serenity js it's Screenplay with Typescript, which is very elegant to use. It's a really nice combination of tools. The reporting is more or less identical. It's the same reporting engine. The screenplay APIs are slightly different, but Screenplay is looks like Screenplay no matter what language you're using. Or even the and the Knight's Python implementation looks like Serenity. All of every Screenplay implementation looks very similar. So there's not too much difference there. It's fairly easy to go between one or the other. So the real choice is what you're more comfortable with. If you're more comfortable with in the Java environment, if you're building spring boot microservices or some backend services. And Serenity JS is probably. Serenity is probably a good choice. And if you're building more in a Node JS environment, or if you're doing a lot of front end stuff or then and you're you've got a Node NPM type pipeline, then Serenity JS is a great choice.

So I guess for the folks that don't know the screenplay, you've been talking about the Screenplay pattern forever. You, I think you did a session on the original Automation Guild, but it seems to be just starting to take off now, even though it's been around for a while. So for the folks that don't know what is the screenplay pattern, how is it different from the page object model?

So we do often compare Screenplay to page objects, but really when you think about it, comparing Screenplay to page objects is kind of like comparing functional programming to encapsulation. It's the screenplay pattern. It's not a design pattern in the same way that a page objects is a design pattern. It's more a paradigm, it's a way of thinking. So it's more like functional programming for automation in some ways because the screenplay pattern changes the way you model your tests or you think about your test. So rather than modeling the user interface that you interact with and then having your test just talk in terms of what the user interface is doing, you think in terms of what are the tasks that the users are trying to achieve and how are they going to go about achieving those tasks. And then at some point you will interact with, you will do clicks and selects and, and enter values in forms and interact with dropdowns and so forth. You'll get to that at some point. But it's part of a task, not part of a page. Because in modern applications the concept of a page is a little bit nebulous. And so with you're really talking about groups of related fields that will change at the same time, that's sort of more what you're interested in. And then you're more interested in how does the user interact with those, those fields or those elements or. And when you model your test in terms of user behavior, you not only get a lot more reuse because you're reusing things blocks that are more natural. Sort of like you build Lego blocks around what the user does in business terms. So does the user. Or if we think simple example, does the user add an item to the shopping cart? Does the user check out an item? Does the user save an item to their wish list? All of those concepts, you can model them as signal objects or a signal task and then reuse them and then you can configure them so that you can reuse them with slight variations. And then you can also. That frees you up from the, in the page object model you are, when you're working a lot with page objects, you're thinking in terms of pages, you're thinking in terms of UI interactions. But if you're thinking in terms of task, you don't necessarily have to have a ui. It could be going through an API, it could go through a UI or an API depending on the context. So you get a lot more flexibility about how you actually implement your tests. So that's the idea of the screenplay pattern. It's a way of testing that models the user behavior rather than the user interface.

And John, what I love about you is like you said, you speak with multiple enterprises, you speak with all types of companies as well, big, small, large. And you've been using screenplay pattern for a while. Last time we spoke was at an automation guild and you. I forgot what the percentage was. But you know, in a real world example from a very large application, you said the screenplay pattern saved like I thought it was like, I don't know if it was 20 or 40% of time, but do you have any examples of how much the screenplay pattern can help with automation, even the creation of it or the maintenance I do.

I had an example that was 50% recently, but literally the real live. I mean once you scale it, the thing is you scale. And when it's a large application you can get that sort of figures because you get a lot, not just the screenplay, but the quality of the task and the components tends to go up because it encourages really good habits and really good design patterns. Really nice code and makes it easier to scale at that level. So the code tends to be more concise and more, more focused. And so you get a lot of. I think a lot of the saving comes from that as well. But I was seeing 50% saving. I don't, I mean that's not. That won't apply for all projects, but definitely one large project we were getting that and I think on one of the first projects we did, which was very like 7 or 800 test cases, that was around 40% saving, I think.

So one feedback I did get about the screenplay and this was early on. I don't know if it's been. It's easier now. It seemed to be complicated or a lot of over engineering almost. Is it because someone maybe didn't understand the implementation of the screenplay pattern? Or does it take a little more upfront cost or time, but in the end you end up saving in the long run.

I think it's like the transition from if you're going between a procedural programming model and you go to object orientated or if you go from OO to functional, it seems complex at first because you're changing the way you think and then. But when. What I found is that when we show screenplay to developers or developers with a bit of experience, often they will pick it up. Or testers with a development background who have seen a few different ways of coding and who have experience with real world applications. Screenplay patterns, effectively a combination of the command pattern and the builder pattern comes down to no o noo patterns and then a functional, fairly functional style of coding. It's very easy to pick up for someone who's already come across that before. And I've also seen. We had one project where it was a lot of quite junior testers, testers who had relatively little automation experience. And we found that if we sat with them for a day and explained the concepts, they picked it up quite quickly. But not only that, we could also it made it easier to scale safely because we could get the less experienced testers to basically assemble LEGO blocks of existing tasks. And then the More experienced testers to actually implement the. The more complex tasks. We'd have tasks like given a customer as a. Given Joe as a high risk customer and then Joe would come from Argentina and have dodgy relations or something like that.

This was a compliance application, so. And then behind the scenes that would create a batch file which would submit a batch operation to a mainframe do so to set up some test data was horrible. But screenplay made a lot easier and what that did was that that was quite a complex, complex interaction to do. But we could use still scale it. The testers with less experience would just use it like a Lego block and didn't have to know about the complexity.

Awesome. I would highly recommend then John, that people do pick up this book. So I'm pretty sure you cover the screenplay pattern in here as well, is that correct?

Yes. There are several chapters on screenplay and explains it right from the very start.

Okay, John, before we go, is there one piece of actionable advice you can give to someone to help them with their automation testing or BDD efforts? And what's the best way to find or contact you?

Best way to find to get in touch is. Well, LinkedIn works well. That's a way to get directly in touch. For the Serenity Dojo training program there I can put a link, a URL where you can get in touch as well and chat or have a. We can have a chat, yeah. LinkedIn's probably the best way to get in directly in touch.

Great. And one piece of actual advice you'd give to someone to help them with their automation testing or BDD efforts.

Focus on keeping the code clean. The what I find is that simplicity and clean code goes a long way and people tend to make the code more complex. Make the code. Yeah, complicated. And when I say complicated doesn't necessarily mean complicated patterns or design or whatever, but just long and convoluted and overthought. There are a lot of like Serenity does a lot of stuff in the background. It's not the all automation frameworks do a lot of stuff that can allow you to code in a more simple way, a higher level way. That's what you want to aim for. Make sure your code actually does what it says and says what it does. And that's kind of the art of writing simple code. It doesn't have to be too simple. It has to be simple enough to do the job it's supposed to do. But the quality of the automation code is sort of the key to keeping things maintainable.

Thanks again for your Automation awesomeness the links to everything of value we covered in this episode. Head on over to testguild.com a449 and if the show has helped you in any way, why not rate it and review it in itunes? Reviews really help in the rankings of the show and I read each and every one of them. So that's it for this episode of the Test Guild Automation Podcast. I'm Joe. My mission is to help you succeed with creating end to end full stack Automation awesomeness. As always, test everything and keep the good. Cheers. Hey, thanks again for listening. If you're not already part of our awesome community of 27,000 of the smartest testers, DevOps and automation professionals in the the world, we'd love to have you join the fam@test guild.com and if you're in the DevOps automation software testing space or you're a test tool provider and want to offer real world value that can improve the skills or solve a problem for the Guild community, I'd love to hear from you. Head on over to test guild.info and let's make it happen.