How to Create Automation Magic with Benjamin Bischoff

Get ready to discover the most actionable end to end automation advice for some.

Of the smartest testers on the planet.

Hey, I'm Joe Calantonio, host of the Test Guild Automation Podcast and my goal is to help you succeed with creating automation awesomeness.

Hey, it's Joe and welcome to another episode of the Test Guild Automation Podcast. And today we're talking with Benjamin all about how to create automation magic. If you don't know, Benjamin's been a software developer and trainer with over 23 years of experience. He decided in 2016 to make test automation his main profession. After working as a test engineer at Trivago for five years, he returned to his former employer Ubisoft as a senior automation engineer in 2021 to contribute to his knowledge of software testing and automation in the field of computer games. So in an interesting ping pong move yet again, he returned to Trivago after five months in order to continue working on various aspects of test automation. He's also the author of two open source projects for cucumber bdd, parallel test execution and reporting. And I just saw somewhere on social media he's releasing a book. Hopefully we'll get into that on karate. Also, he's a conference speaker, so if you ever see him at a conference speaking, you need to go up to him because he's also a super cool magician. So he does some really cool tricks. I think your mind will be blown. Really excited to have him on the show. You don't want to miss this. Check it out.

This episode of the Test Guild Automation Podcast is sponsored by the Test Guild. Test Guild offers amazing partnership plans that cater to your brand awareness, lead generation and thought leadership goals.

Get your products and services in front.

Of your ideal target audience. Our satisfied clients rave about the results they've seen from partnering with us, from boosted event attendance to impressive roi. Visit our website and let's talk about how Test Guild could take your brand to the next level. Head over to testguild.info and let's talk.

Hey Benjamin. Welcome to the Guild.

Hello. Thanks for having me.

Awesome having you. I always botch bio. So is there anything I missed in your bio that you want the Guild to know more about?

Yeah, my bio is white, hard to read. Also for me, this interesting ping pong move that that's still in there is not really that important. I just noticed what's more important is that I have been in automation for quite a number of years and that's also part of the reason I came back to Trivago. To stay in this automation position and test position, because that is what I want to do.

Why?

That's a very good question, actually. Because it mixes everything I like about software. It mixes development aspects, it mixes software architecture. You have so many insights into so many different aspects of software development where as a pure developer, you typically focus on one specific area. But as a tester, you get in touch with a lot. So you learn a lot.

Love it. I totally agree. How much carryover is there with. Do you have a background in development that makes you test automation engineer? How many of those skills are applicable as a test engineer?

I think actually a lot. I think what's cool about coming from a development background is that I kind of speak the language of the developers, so. So I can be in the middle of QA and developers, so to translate a little bit sometimes. Also I bring software craftsmanship principles and clean code principles to test automation code, which in my books is pretty important to keep your solutions maintainable and scalable.

Love it. So also being a developer and a tester, do you find when you have discussions about developers, maybe they say, oh, you can't test that way, or like it's more of an education process? Well, actually I know, because I'm a developer too, that you can do it. And here's how. Do you ever get yourself in conversations like that?

Yeah, that happens. I have to say, it used to be quite hard to be in QA as a developer because the way of working is different. If you come from a pure development background, it's like you're very biased. And I know from my former position as a developer, I had my biases towards qa and I always keep telling the story that we even at one time in a team knew about a rather serious bug, but we didn't fix it because it was too complex. And we said, as long as QA doesn't see it, we don't have it. And yeah, I tell this because this is not how I work anymore. So I learned a lot from qa and I hope the QA team learned quite some things from me too.

Great. So developer background.

A lot of times testers are told.

Oh, you need to use the same tools as the developers, you need to use the same languages as the developer. Since you are a developer and you're our tester. I don't mean to keep hopping on it, but it's a different perspective. What are your thoughts on that? Because I know one of the tools that you wrote a book about is not necessarily seen as a developer tool.

Right. So. Oh wow. Actually two parts to this. So the one part is I believe that it does not matter what tool you use and what language you use for testing, as long as the development team is not the only entity to do the testing. Like for unit tests. I think it makes a lot of sense to have the same language language as the application code. But if you write end to end tests or API tests, I don't think it matters that much. It matters more what problem you want to solve and what tool fits the best. And I think enforcing the same language devalues developers because the concepts are the same. I mean, if you jump from language to language, there shouldn't be that much problem in test code because usually it's not super complex code. Especially if you use tools like Karate that are really easy to get into.

Yeah, I guess the argument always was, well, developers in a flow, they want to use the same ecosystem, same tooling, don't want to leave their ide. But you know, I haven't been a developer, so I don't know if that's actually a concern that you would have as a developer day to day.

I don't think it is really a concern. I mean, for us we use a lot of different tools and different languages depending on the problem and we try to analyze the problem first and then choose what fits best. And that it's the same language is not one of the top criteria.

So going from developer to testing is kind of unique. Sometimes testers, they feel set off in testing so they can become a developer. I know a lot of folks that love testing. I love testing, spend my whole career. So if someone's been a tester their whole life, how much development skills should they have? Or is that something they should be concerned with? Because I guess if they have testing, that's probably the hottest skill to have and they'd probably already have it. So how much developer skills do you think a tester should have?

Depends on what is being tested. If you are like a super good exploratory tester then, and those are still needed and are needed even more now with all the AI stuff coming in, then you can have a great career without any coding. But if you notice that you're doing the same things over and over again as a tester and you want to change that, then it's a good time to start to learn about automation a little bit. And also don't learn the language first, learn some of the concepts first and I think the rest just develops. I mean, it was the same for Me, when I got into testing, I didn't have any testing knowledge and I had to acquire this knowledge too, which was really hard at first. But yeah, you can be an automation engineer or software development engineer and test. It doesn't matter from which side you enter this path.

So I like how you mentioned earlier how you like to bring clean code principles to automation. So what does that mean? So if someone's like clean code, what?

Yeah, I mean if you are on social media, LinkedIn, you see quite some examples of test code and also if you check GitHub repositories and I noticed that a lot of times the code is not as good as it could be. And if you don't have that much experience and you just copy and paste code like this and you extend your test suites at one point, it will become a nightmare to maintain. And if you want to change something, you have to change it in 100 different places. So I think it's really important to keep this organized and apply some software architecture principles to it so you can maintain it in the future or some other people can maintain it and understand it.

So I guess the follow up is what kind of software architect principles should someone apply that you think for a test is relevant to automation code? Because I know not all code, I mean code is important, but automation code. I guess there's some things that may be overkill where in production it makes sense. Maybe I'm wrong.

No, you, I think you're completely right. It depends on what the code is doing. If it's a thing that is doing one specific task over and over, it doesn't matter that much that it's super clean as long as it works. But if you have, if you write a test framework that is used by many to write test cases, then it's important to keep it really concise. I mean, the simplest thing that testers encoders should apply to their code is proper naming. That's where it all begins so that people can follow what this piece of code is doing. It's often said that the best code is code that documents itself. And if you name it and keep your methods very short and concise and make everything just do one specific thing, that's a great start already.

So I've said this multiple times. Probably most of your career is done debugging or maintaining tests than writing tests. So if it's not written clearly, it could become a nightmare. It seems so, even though it seems trivial, if you name, even if you wrote the code yourself six months later, if you didn't name things correctly, you might be like, what did I do here? Right.

Exactly. And that has happened to me when I look at my own code and I didn't understand anything. Yeah. So it's not only for others, it's also for yourself. Yes.

So we met in person at Selenium Conf, and I believe you actually did a session around code smells, which I think goes along with clean code. So, once again, for folks that may not know what is a code smell.

A code smell is something that makes you. Yeah. Give you a strange feeling when you see it. And it can point to something that can be wrong with the code. It doesn't have to be something dangerous, but it's just something that gives you a feeling like there's something fishy about this piece of code. And my session was about how to pinpoint those things, how to name those things properly when you want to talk to someone else about this and what could be problematic about this.

So can you give some examples?

Yeah, there is, I think there are 21 code smells I presented. The easiest one is probably the long method smell, which means that you have a method that does more than one specific thing. Or if you name a method, I had the example of a safe customer method that not only saves the customer, but also gives you some HTML back that displays something about the customer. So this method would do something that you wouldn't expect from the naming. So this is a method that's called long method. Not because it's long, it's a long piece of code, but it just means it does too much. That would be the simplest. Nice.

Another one that stuck out to me is, I think it was broken windows. Broken windows?

Yeah. Broken window is not a code smell. It's more something that happens to software if you don't fix stuff. So if you imagine a house with a broken window, if you don't repair this window in the next days, somebody might come and look at this and say, oh, this has been broken for months, so let's break another one just for fun and just throw some garbage into the hallways. And then in the end, you just have a house that is completely broken and unmaintainable. So what this, this tells developers is you should fix mistakes early and you should pinpoint bugs early before other people make the same mistakes and ruin the whole code base.

So it's not a really good segue here, but. So I know you're writing a book on writing API tests with karate. There's some controversial version around karate for some reason, because a Lot of people say it actually creates code smells, and I think it's because they don't understand what it is. They think it's a BDD framework, which I don't think it is, it just looks like it is. And then when people see that, they automatically make assumptions. So I don't know where I'm going with this, but do you have any thoughts on Karate and how maybe people may be getting confused with it not really being a BDD framework? Or maybe it is and I'm just wrong.

Yes, you said a lot of true things about this, because when I first encountered karate, I thought the exact same thing. It looks like bdd, it looks like Gherkin language. And when it was started by Peter Thomas, it was actually using this part from Cucumber, the Gherkin parser. But Gherkin and BDD means that you describe behaviors of your applications that you want to test. But he took this concept and made it more technical because describing APIs with this behavior driven approach is not really feasible. I mean, you wouldn't say I send the appropriate data to this endpoint, but you would describe what is the exact data and what is the exact endpoint and what is the structure you expect. So it does make a lot of sense. The controversy just came from this format of how the scenarios look. And he also wrote about this himself and said karate is not bdd and it is certainly not bdd, but once you get over this mental block that you have, it's really fun to work with.

So you mentioned, you know, it's an easy tool to use earlier when we first started. And as a developer, would you developers.

Use a tool like this?

Like, can you give it a little example maybe how you would write a test with Karate and how maybe someone would utilize if they're not a tester.

With Karate, you can just. If you know tools like Cucumber, you will typically write test scenarios. Like the user goes to the website or the user logs in and then he enters his order. And in the end, I don't know it costs that much money. With Karate, you use the same principle, but you don't write all this code behind your test scenarios. So what Peter Thomas did, he used this format of the scenarios, but without what's called the glue code. So you don't have to, for every step that you use in your scenario, you don't have to write the appropriate code that translates what you write in Gherkin to actual Java code or JavaScript code. But Karate does it by itself. So you don't really write code, you write more test scenarios. But still you have to understand exactly what you're testing. And so it is definitely a tool that can be used by developers as well.

So in the book, the book isn't released yet, so I haven't been able to actually look through. I just looked at the high level information.

Do you cover any clean code principles.

Applying it to karate, or is that completely a different domain you didn't touch?

That is a little bit of a different domain. I talk about how to keep your scenarios concise and yeah, you could say clean so that they're easy to maintain. But since you're not dealing with too much with real source code, those clean code principles don't always apply. I mean, I have chapters where I also extend karate by JavaScript code and by Java code. And there I'm really careful to not violate any clean code principles. But it's not an explicit part of the book.

So I. I think you also touched on. Maybe I'm wrong. Like I said, I just read the high level overview. Using karate for performance testing.

Yes.

Is that possible? What does it use? Gatling behind the scenes. How does that work?

Yes, it's using Gatling. So that's another thing that is really, really nice about karate because you can just take your test scenarios that are already there and by just modifying how you run them, you can use them as performance tests using Gatling. So you get all the Gatling performance test reports. You can also explicitly choose which of your scenarios should be used as performance tests. So you have tons of possibilities.

It's almost like magic. So that's my segue. Speaking of magic, you are a magician. I saw a session that I loved could be like your signature kind of keynote. I think it was like nine Magic Principles in Software Song when you actually were talking about you were doing magic and then you were applying it to software, which I thought was so cool. So for a little background for people like as a magician, what principles have you found kind of go or correlate with testing principles? Maybe we could dive into that really quick. Sure.

I mean this session that you have seen is called Smoke Tests and Mirrors.

That's what it is. Yes. Yes.

And yeah, it's becoming one of my signature things for sure. It all originated in some blog posts I did comparing magic and software principles. And you have some principles that are well known like misdirection that can directly translate it to software and test reports and also stack traces in code that can misdirect you heavily depending on if they show too much or too little information, but also things like clarity of effects in magic and clarity in UIs, in APIs, in everything that a user touches on your software. I mean, there is so much that you can compare. And part of the reason why I did this was not only to do this, but also to show how you can apply knowledge that you have from outside your profession to your profession.

So I love doing this to other people, giving them work, but I think this would make a great book. So why did you choose doing a book on karate rather than this one, which I think would be like such a cool evergreen type of book.

Oh that. I have never ever thought about turning this magic and software thing into a book. But now that you say it, I'm thinking about this.

I can actually see like the copy, the design. It would be wild, I think.

Oh, that's a great idea. The reason why I wrote a book on karate was very simple actually. I was approached by the publishing company and they asked me if I could write a book on. I won't disclose the topic here, but it's a topic that A has a lot of literature already and B, I'm not an expert in. So I told them I'm using this test framework and there is no literature about this yet I would feel comfortable writing a book about. And that was karate. And they said, yeah, after some research they said, yeah, there is nothing yet. I just thought something is needed because it's a great framework and there are so many questions on stack overflow about it. So I just thought there must be something. So I wrote it.

How long did it take?

I started in November, so it took half a year roughly.

It's pretty quick for a book, technical book. That's awesome.

Yeah, I have a really tight schedule, so which is cool because I am a procrastinator by heart. So if you have really tight deadlines for each chapter that really gets you working.

When's the book going to be available?

The book is coming out May 19, so it's on pre order right now at the time of this recording.

Yeah, I'll definitely have a link for it and people should definitely pick it up. Okay, Benjamin, before we go, is there one piece of actual advice you can give to someone to help them with their automation efforts and what's the best way to find or contact you?

So the advice I have for anyone starting with automation is talk to people first, don't start with an automation solution, then try to force it on people. Because that is exactly what happened to me and then you have the risk of this whole automation project just falling apart. You have to understand the problem first. You have to understand what people are struggling with to be able to come up with a good automation solution and then start very, very small. That's my advice and you can contact me the easiest is on my website. It's called SoftwareTester Blog and this is also where I write occasional articles. And there you also have the list of all my social media links and everything.

Thanks again for your automation awesomeness. The links to evident value we covered in this episode. Head on over to testguild.com a444 and if the show has helped you in any way, why not rate it and review it in itunes? Reviews really help in the rankings of the show and I read each and every one of them. So that's it for this episode of.

The Test Guild Automation Podcast.

I'm Joe. My mission is to help you succeed.

With creating end to end full stack automation awesomeness.

As always, test everything and keep the good. Cheers.

Hey, thanks again for listening.

If you're not already part of our.

Awesome community of 27,000 of the smartest testers, DevOps and automation professionals in the world, we'd love to have you join the fam@test guild.com and if you're in the DevOps automation software testing space or you're a test tool provider and want to offer real world value that can improve the skills or solve a problem for the Guild community, I'd love to hear from you. Head ON over to testguild.info and let's make it happen.