Writing Great Specifications with Kamil Nicieja

Welcome to the Test Guild Automation Podcast where we all get together to learn more about automation and software testing with your host, Joe Calantonio.

Hey, it's Joe. And welcome to another episode of the Test Guild Automation Podcast. Today we'll be talking with Camille all about his new book Writing Great Specifications. Camille is a seasoned engineer, architect and project manager with deep expertise in Gherkin and Specifications by Example. He is also the author of the book Writing Great.

Using Specification by Example and Gherkin by Manning Publishing. So get ready to discover how to use BDD for maximum results for your team. Listen up.

The Test Guild Automation Podcast is sponsored by the fantastic folks at Sauce Lab. Cloud based test platform helps ensure your favorite mobile apps and websites work flawlessly on every browser, operating system and device. Get a free trial. Visit testguild.com sourcelabs and click on the exclusive sponsor section to try it for free for 14 days. Check it out.

Hey Camille. Welcome to the Guild.

Hi Joe. And hi everyone to who is listening right now.

Awesome. Thank you so much for joining. Before we get into it, is there any missed in your bio that you want the Guild to know more about?

Maybe I could update it a little bit because it was a while since I actually started working on the book, so some new things that appeared since then. I'm currently working on a startup which is called ADA and it's a real estate startup and actually one of the examples in the chapters in the book is based on on our work on discovering features and bugs and requirements that are connected to the startup. So writing the book and working on this new application was very interconnected and I was happy to do both.

Awesome. So I'm definitely going to dive into that because I think the new company does AI things for artificial intelligence. Is that correct?

Yes, what we're doing, we are building algorithms using artificial intelligence machine learning to analyze real estate. So what we're doing is we are estimating prices, estimating demand, estimating supply, analyzing locations and so on. This is all available through APIs. So any company that is interested in real estate can just plug into our APIs and just start using the algorithms right away without having to develop any technology on their own in house.

Awesome. So this is probably a little early for this question, but are you actually using BDD then or Specifications by Example with your team as you develop this cool technology? Nice.

Yes. Both BDD and Specification by Example and Gherkin were used during development and our documentation process and specification process are oriented towards building living documentation and testing as well as developing and Discovering requirements through bdd.

So, so here's a dumb question. Is there a difference between BDD and specifications by example? Because you tend to use them differently, I think.

Yeah. So it's one of the questions that is answered in the book. I think that generally people are focused on the names and the processes, but what is worth talking about is the common things about all these practices. And I think that, you know, if we would dive into it, I think that for example, acceptance testing is more focused on the testing part. Right. So if you dive into this topic, you will discover much more things about the technical aspects like how to do automation, what is the best test, the testing tool, how to do things in the correctly technical way. So what to do to not make your test brittle. And specification by example is more business oriented, but I think, you know, it's the same family of tools, the same family of practices, but the focus is laid differently in these sub communities of this bigger branch.

Awesome. So writing a book is very difficult and working for a startup I think would be extremely difficult. So why did you decide to write a book during all this?

Yeah.

This is a little bit interconnected because it just happened at the same time. The idea for the book was actually conceived in my previous job which was a web development shop, an agency where we were working with clients from all over the world. And this is where I first discovered specification by example. This is where I first discovered Gherkin. This is when I first started working with Cucumber. But later on I thought that.

It'S time to move forward and I started thinking about the new gig and the idea was to become a consultant so that I could start teaching these practices that we discovered in the previous job to new people. And this is something that I started doing. This is also where the initial idea for the book was started. And then it just morphed into a different business because I met a new friend who had a different idea of what we could do and we just started doing that. But you know, the book was already project that was in the works so I wanted to finish that. I wanted to in a way sum up this period with working with specification by example. And I thought that this was the best way to do that because it would be a pity to just lose all that knowledge, all this, all these ideas that were considered there.

Great. So it sounds like maybe I'm wrong. Sometimes when you work for a company, something works for them, but when you go to another company just doesn't take off. And it's totally different. But it sounds like you were using specification by example at your old company and then you implemented at the new one as well. They seem like totally different domains. So you've been successful, able to implement them at both companies?

Yeah, so that's right. The first company I was working at was about 50 people. And then of course the startup that I was founding, it was about three to five people, depending on the, on the timeline. So these are two very different scales. But it was working both in the startup and in the company. And moreover, the company that I was working at called Monterrey, was a web development agency. So we are working for clients who are also very different sizes. For example, one of the clients could also be a startup, but at the bigger stage and they could have, let's say, 20 people. Right. But another client was big enterprise company that was employing thousands of people all over the world and we were working with them as well, using these processes. So this was a great playground to just try it out in different contexts. Right. So that was a great experience for me. And this is where the most of the practice is described in the books come from.

Nice. So why do you think teams need specification? I know you describe this in the book, but I like to set the stage for why. Why do you think there was a need for this?

So I would say that the most important thing that teams need to understand is that there is something called information asymmetry. So this is a situation where different parties have incomplete information. For example, I as a person in one department can have some part of the call thing that we're building, but another person in another department might know something else that I might find useful and so on. So specification by example is a way to bring everybody to the table. Bring the business people, the testing people, the developers, the UX designers, UI designers, and so on. So if we bring everybody to the table, we bridge the communication gap. And this is, in my opinion, the most important aspect of specification by example. Creating complete requirements, or as much as they can be complete in the real world through great communication and then having it written down, automated and maintained over the course of time?

Yeah, BDD done right. Sounds awesome, but I know especially in the testing industry, BDD has a really bad rap. So are there any common pitfalls you see when people trying to use specifications that maybe they're not seeing the benefits of the communication that you keep bringing up and maybe just something that they like, a bad practice that they fall into that avoids all the benefits you get from using VDD and specifications by example.

I would say that some People forget that it's just an utility, a means to an end. Right. The goal is communication. And some people, and some companies, some departments just sometimes focus too much on the way they learn to use the process, on how they do things, instead of just trying to communicate and then be being agnostic about the tools and the processes. Right. There are many different ways to do these things. And the way we learn to do it doesn't have to be the best way in this particular context with these particular people. For example, in one of the chapters in the book, I talk about the situation when we had a team. One of the clients was they didn't have any technical people on their team. And it was difficult because sometimes Gherkin, Cucumber and Specification by Example can get a little bit too technical. Right. This is a language that can be readable by the business, but it's also connected to automation and many different technical tools. So it can get difficult for people who don't have any experience with that. So what we did was we just communicated through the tools that they knew, and in this case it was Excel. Any person who is older than 18 can use Excel. And this was what we are doing. The examples were written down in Excel, then we took them out of the Excel and we just made the specifications through. We wrote them ourselves based on the Excels. So this is something that we were doing because we were agnostic about the tools. Right. And I think that this is the biggest pitfall, focusing too much on the technology, focusing too much on one particular way to do things.

Yeah, I think that's a great point. So how did you get developers to embrace this? So I guess in my experience, I work for a very large enterprise, maybe it's not a good example because I think it was very toxic there. But how did you get developers to actually embrace BDD or Specifications by example?

It's actually an interesting topic because I got them to embrace BDD because I was one of the developers. So in the beginning we didn't have a lot of testing practices. This was one of the things that we are doing poorly. And there was this huge effort in the company to fix these things. And as usually happens in such situations, many different initiatives were taken during that time. Somebody started using this tool, somebody started using a different tool. And I and some of my friends at the company just started using Specification by Example and Cucumber and Gherkin especially, and it just took off. We had successes using it. We got other developers on board because we showed them how we did things. We Got designers on board because one of the people who was interested in this beginning the process was also designer. He was interested in having a say over the requirements. So he was interested in getting to this table where the discussions were being held. We did that with specification by example. This is how it took off. So my experience in a way is very mixed because I started as a developer but then I was, my job was changed to product manager and I was on this more business oriented side of things and I still kept using specification by example and it kept working. So this is how it grew within the company and this is how the initial adoption started.

So far you've talked a lot about communication. You didn't mention automation once. But what does automation mean in the context of bdd, in your opinion?

Yeah, so automation means that every conversation, every scenario, as it is called in Gherkin, is written down and then this text artifact is automated with code. So text is always connected to code, but maybe not always because it doesn't have to be in some cases, but this is always huge hole in the process. So in the ideal example, the text is always connected to the code, which guarantees that the text is always up to date. Because if the text is not up to date, then the code breaks. Right. Because every scenario has a testing, has a test code below it. Right. So it guarantees that the specification suite is always up to date and we don't have situations when something rewrite in the specification or living documentation is just plainly wrong, outdated and changed four times since it was written.

Are there any anti patterns when using specifications for automation?

Yeah, generally when it comes to automation, the biggest anti pattern in my example, in my experience, is focusing too much on the technical side of things because I was a developer and as a developer I know that sometimes it's easier to write a test that is easier to write and easier to maintain over time, but it's actually impacting how the scenario, how the specification is written and then it becomes unreadable to less experienced people or to people who are not technical. For example, some specifications written in Gherkin I read had some fields from the database in it. Right, because there was a rush in the team, there was a deadline and they were just trying to make it work and write it as soon as they could, as simple as they could. And this resulted in a specification that was actually unreadable to anyone who wasn't a programmer because it was talking the language of the technology. Right. So what I would say that when it comes to technical aspects of automation, you can find A lot of resources that are already available. There are like a hundred books that are written about unit testing, about automation, about automated acceptance test and so on and so on. Right. So this is practically a field that is I would treat us more or less solved people can write good tests thanks to td, thanks to bdd, thanks to years of books, articles, tutorials and so on. But what is often forgotten is this core practice communication that we still have to care about this non technical aspect and that this technical aspect of automation can influence, for example, adoption. If we focus too much on the technical aspects, it might result in specifications that are too technical and then the business side of the project will just reject it. So this is a risk that we're facing. And without the business side of things, without the business people, we won't be able to do the process at all because it is about bringing all the different competencies in the team to the table.

So I've seen two different implementations of bdd. One is really business focused. So given I'm a radiologist, when I. When a patient checks into a hospital, then an exam is created for them and then you have other ones where like given I go to this URL and I click on this button and I type in this, this patient name. Where do you draw the line? Either one incorrect or correct or what do you recommend to folks?

Yeah, so I'm definitely with this business oriented way because as I said, there are many different tools that are available to developers or testers that can answer their need to test things like that. Like in the second example that you said, like I go to the URL, I click on this button, I fill in this form, you can just write acceptance tests and don't call it BDD because it just not using the process to its fullest. Right? But when we focus on this business language then actually it's a space to learn something new. Because we can start using these examples, these scenarios, not only to describe what the user interface of the application is doing. This is actually simple because everybody can see what the application is doing at any time. You could just record the video or prepare a folder of screenshots. Right? But when you start talking about the business, about the requirements, about the outcomes, about the impacts about the people who are using this product, this is where the interesting things start to happen. Because this is where somebody can for example, say okay, I think that this person, for example the doctors from your example could use this differently because you're not focusing on the particular button or the particular form, but you're Focusing on the business flow so you can start discovering things. You can start using specification by example, not only to test, but also to do something much more important, which is discover the correct requirements that make the software successful.

So what's the life cycle of a specification? Do you have like an amigos meet together, three amigos, you write out the conversation and then you automate it and it goes into your CI cd. Like what's your typical life cycle, I guess of an executable specification?

We start with understanding business goals. This is one of the most important things, I think, that is often forgotten. This is the moment where we do things like start writing user stories, try to do some metrics, success metric that would tell us that this feature that we implemented actually doing something good to the business, or was it failing? Do we have to redo it? And so on. So the first thing is to get everybody to the same table and discuss goals. Because if we don't have goals discussed, then we don't really know what to do next. We can just talk about the technical solutions, like, for example, we want this feature implemented right, but you don't know why actually the business wants this feature to be implemented. You don't know what it needs to achieve. If you don't know what it needs to achieve, then you don't know how it should work. Because it's impossible to evaluate this particular solution that you are working on. So the first thing is goals. Another thing, the second thing is analyzing requirements with examples. So since you have the goals, you can start deriving requirements from goals. So for example, you can say that to achieve that goal, the application must do this and the users must be able to achieve this outcome. Next, if you have the outcomes and requirements written down somewhere, you can start illustrating them with examples. We can take a real life example, for example, taken from the business people or the end users, the people who are actually using the application. And then you can just illustrate the requirements to understand them better or to reorganize them further. Because the example can show you that this requirement actually has to have a few more acceptance criteria. It has to be done differently because the real world users are using it different. So analyzing requirements with examples is the second step. The third step is deriving scenarios for examples. So you have the examples, you have the user story, you have the goals, everything written down. Now you can sit down and write it down. And the three amigos you mentioned, this is something that can happen either before or at this stage. So depending on how your team is organized, it would be Good to have everyone within the discussion as soon as possible, because it helps avoid reworking things in the future because everybody can talk about their ideas or what's wrong and can point out that something is not working as it should as early in the process as possible. So the Three Amigos meeting can be helpful at this early stage, but it can also happen at this stage when the initial requirements are already somehow loosely defined. And this is the moment that we are using scenarios and gherkin and cucumber to flesh them out. So, so we can start working with them, we can start resizing them, we can start making them more and more real. Right? So this is the third stage of the process, and then the work with scenarios continues. So we are adding new examples, we're refining scenarios. I understand that usually the first drafts of scenarios are rather rough and they are not a great read, as any first draft is, but this is something that you're just working through with the team. You're adding scenarios, you're removing them. Some development is going on during that time already. So you know that this thing that you thought is going to work this way, it cannot be like that. So you have to change it. So you go back to the drawing board, you rewrite the scenarios, and so on and so on. This is the moment when we get to implementation and hopefully release, production, release. And this is the moment when we start iterating specification over time. So unlike the textual specifications, specifications by example don't just end their life at the moment that the feature is released. Specification by example continue to be used after the release because we continue to use them as living documentations. The code, the testing code is connected to the specification text, so we can easily see what is being changed in the application, what needs to be changed in the specification layer. And the automation layer can influence the text so it is kept up to date. This is the same stage when validating the specification suite happens. So we are using the testing code and scenarios that we're reading to continuously validate our application, our product. And this is when, as you said, continuous integration, continuous delivery comes into place. We can just use this specification suite as another testing suite, for example, unit tests, or any other test we are already using in this product.

Great. And so this book's broken in two main sections. The first one is writing executable specifications with examples. You have all these chapters around more like creating your executables and creating your specifications. And then the second part is broken into managing specification suites. So so far, just so folks listening, I've covered pretty much every chapter so far in that first part. But if you want to hear more about how to manage specifications, you definitely want to go to the show Notes of this episode because I'm going to have some links to get a free book. If you leave a comment below and we choose it as one of the comments, that's going to get you a free copy of the book which is writing great specifications. If you want to grab that book, make sure you head over to the show notes and we'll have five free copies we're giving away based on your comments. Is there any questions you want me to ask for? I asked my very last question.

No, I think. I think everything was covered. The most important part I want to say to people who are listening to this is that this is a different book than the technical books about testing. Right. Because this field is pretty much solved. There are many great books that were already written about the testing practices, the technical practices. As you said, there were also some shortfalls of the process discovered this way. Brittle Test, Selenium or automation not being the best way to do everything. This is not something I was interested in. I was interested in the business, you know, bringing everybody together using these tools in a different way that they were used. And actually if you look at the book, the any testing code appears only in the last chapter as an example. So this is very, very focused in the business and very focused on getting the right outcomes out of the process. And I think that this was the part that was missing on the market because everybody was talking about how to automate these things, but nobody was talking about how to write these things so that anybody would read them. And this was the missing part of the process.

Awesome. Okay, Camille, before we go, is there one piece of actual advice you can give to someone to help them with the BDD specification by example efforts and what's the best way to find contact you or get a hands on your book writing great specifications?

Yeah. So if you want to contact me, I think the simplest way right now would be to just go to the mining forum. There is a topic there about the book and I'm on the daily basis I'm looking at it and just answering people's questions. So this is the best way, I think. But the actionable piece of advice unconnected to this one thing would be just start. This is a very forgiving tool to the beginners. As you said, it can be easy to do things wrongly sometimes, but it can be also very easy to start and then just fix it over time and since this is my focus in the book was non technical I think that and the angle I was using was communication based as was often said during this interview. I think that you know as soon as you start communicating using these practices, even one, even then add another and another, you will start using this in a way that can be better over time and start getting the benefits.

Thank you Camille for your testing BDD Awesomeness. For links to everything and value we covered in this episode, head on over to testguild.com a321 and while you're there, make sure to click on the Try for Free Today link under the Exclusive Sponsor section to learn all about Sauce Labs awesome products and services. So that's it for this episode of the Test Guild Automation Podcast. I'm Joe. My mission is to help you succeed with creating end to end full stack Automation awesomeness. As always, test everything and keep the good. Cheers.

Thanks for listening to the Test Guild Automation Podcast. Head on over to testguild.com for full show notes, amazing blog articles, and online testing conferences. Don't forget to subscribe to the Guild to continue your testing journey.