Testing Web APIs with Mark Winteringham

Welcome to the Test Guild Automation Podcast where we all get together to learn more about automation and software testing with your host, Joe Calantonio.

Hey, it's Joe. And welcome to another episode of the Test Guild Automation Podcast. Today I'll be talking with Mark Winterham all about his brand new book testing web APIs and also he's a testing expert automation expert. So I'm sure we'll dive into other things as well. If you don't know, Mark is the dojo boss at Ministry of Testing, one of the number one resources for testing that I know about that's been around for years. Definitely check them out. He also teaches many aspects of software testing and he supports and curates testing training from others in the testing community. He's also the co founder of Automation and Testing and Ministry of Testing Essentials Training Pathways. As I mentioned, there's a lot of great things going on in his Ministry of Testing and Mark is one of the leading driving forces behind it. So definitely show him some love after this podcast is over. Before we get into it though, you definitely want to check out this episode, so stick around. Listen up. The Test Guild Automation Podcast is sponsored by the fantastic folks at Sauce Labs. Their cloud based test platform helps ensure your favorite mobile apps and websites work flawlessly on every browser, operating system and device. Get a free trial. Visit testguild.com sourcelabs and click on the exclusive sponsor section to try it for free for 14 days. Check it out.

Hey Mark, welcome to the Guild.

Hey Joe, how's it going?

Awesome. Great. Great. Yeah. Great to have you back on the show. It's been a while so it's great to catch up. So first question, I guess it's always cheesy, but whenever I speak to an author, a soon to be author, writing a book to me takes a lot of time. I self published mine, it took me a year, you know, years ago. So why, why write a book?

I think there's a couple of motivations. One as a purely ego driven thing, is that there is a law in the UK that basically says that any book that is published a copy has to be kept in the British Library. And just the idea of having a copy of with my name on it in the British Library would be really good. But to be honest, like I never really planned to actually write anything. I was watching this gaming stream so the, the Escapist and there was just a live stream of some game that was being played and the guy was playing it. Yahtzee Croshaw. He's published, author and Someone asked him the question of like, how do you write a book? Like how do you get on with it? And he just went a page a day. So I just thought I'll give that a go. You know, cheesy sort of lockdown, lockdown goals sort of thing. So I started writing a page a day. I actually realized I just enjoyed it. I enjoyed it. I love teaching and this was a new and interesting way to teach. So it's the combination of those two things. So I actually, I wrote a couple of chapters before I even sent anything out to any publishers. And then I only emailed out publishers thinking, well, if I get an email back saying no, that's enough. And then here I am now down the line, signed up with Manning, the book's half done. Yeah. And I'm sort of slowly chipping away at it and stuff. So yeah, it was never planned. But I think it's a thing that I've discovered and it's a thing that I've discovered that I actually enjoy. So I think that's kind of the main thing. It's the process of writing it that's really been quite exciting.

Nice. Maybe I'm full of our profession, but I would think it's actually harder to write a technical book than a novel because you're not just free, willing, typing away. You need to have some sort of structure. The technology always changes, so you need to come up with an example. So to me it's also a lot of hands on work, making sure the examples and everything work. Is that how you approach the book or am I thinking overthinking it?

No, no, you're absolutely right. I think that I wouldn't say one is harder than the other. I would say that they are different. So again, like it was just this weird serendipity moment of watching a few YouTube videos of other things outside of the tech world. This lady called Lindsay Ellis, who just released her first book and she talked about publishing as a fiction author and, and she talked about how from technical books, educational books, you have to have your plan up front, but you don't necessarily have to have everything written, whereas with fiction you kind of have to have your book written first. So they're the big things. But yeah, what's really interesting for me with the dojo boss side of stuff and the learning stuff that I've been doing with Ministry of Testing, so identifying learning outcomes, working activities, what sort of material you teach to achieve those learning outcomes, all of that mindset actually sits within the publishing world as well. Some of the conversations I've been having with my development editor are very similar to the conversations I have with people who are developing stuff for us. We've learned a few things from it. I've certainly learned a few approaches about how to be a bit leaner, be a bit more agile with content production. But it's also been quite validating as well to know that what we're doing at Ministry of Testing as well is similar to other, you know, established professional content creators in different places. So, yeah, they're definitely different.

What I also love about Ministry of Testing is it seems like you have a very active community. It's like you guys are like, it's almost like a cult. So I would assume you get a lot of good feedback about APIs. And so, like, when you're writing the book, are you thinking like, oh, I remember someone asking this question in our community, or we get this asked a lot, or are people always, you know, something like that?

Well, I feel for a starter, I should just clarify that we are not a cult.

No, you're not a cult. No, in a cult. In a good way.

All the good things of a cult.

All the good thing about the cult we'll rebrand.

Yeah, well, yes, yes. So the way that we. The way that we want to work with Ministry of Testing, we've been sort of moving this direction for a long time, is that a lot of the teaching, a lot of the learning and stuff is through the community. So, yes, there are things that I run as training events, there's things I do with Richard with automation in Testing, and they're closely aligned with Ministry of Testing. But crucially, we're trying to do ourselves out of a job. You know, we want other people to be producing that sort of content. And that's through, you know, the conversations on Slack, the Club, that sort of thing as well. But actually, like, for my book, I've been quite cagey about it. Like, I haven't really, until it sort of came out on early access, I didn't really tell very many people because I partially didn't want to jinx it. I didn't want to be, hey, I got a book out coming out, and then it's like, oh, I've only written a chapter in six months or something like that. So I wanted to be careful with that. Also, there is a certain element of the book is separate to Ministry of Testing, like my experiences and my journey is integral with Ministry of Testing, and that exists in the book. But, yeah, the book will be shared within the testing Community through Ministry of Testing, just like anyone else in the community would. There's no sort of. It's important to me that there's no sort of favoritism involved and stuff. I'm just one person sharing some ideas that hopefully will help others with things like API testing and strategy. Really?

Absolutely. I guess what's really interesting that you said you were promoting it. Were you shocked that when you saw at number seven? I think I started manning.

Yes.

Top ten list and I was like, oh my gosh, this is awesome.

So it was number six.

Number six.

Then it disappeared and then it's come back as number seven because I had my first week sales and then I think things dipped because of the holidays and then it's come back up. So I'm astonished, really. I'm really sort of pleased to see the response so far. I mean there's only, there's only two chapters available for people to read. So to get the level of interest that I've got so far speaks volumes to me that I'm going down the right path. It's an interesting challenge working with. I think a lot of technical publications are more focused towards development than testers. So some of the conversations I had initially were sort of more like. Like I'm trying to be more expansive about the ideas of testing as well. Not just about API testing, but just testing in general. But a good amount of the audience for a lot of these publishers are going to be developers. So it's exciting to see them jumping on board as well and seeing if they agree or disagree and having that sort of discourse as well. But yeah, it's cool. It definitely helps with the imposter syndrome, that's for sure.

Yeah, no doubt. Yeah, Congrats. I also kind of excited about this. Get your view on it. Why do you think web API testing is so popular now? Because to get your book at number seven on a topic like web API testing with all the type of topics Manning covers, they seem very developer focused. So to me it excites me because it seems like it would point to there's a lot of interest in API testing now. Am I wrong?

No, but I think it's two parts. I think there is a lot of interest in API work, but I think there's a lot of interest in testing as well. Like, you know, I think if you compared the discourse around testing and quality five years ago to now, I think quality is starting to gain more traction within teams in general. There are still going to be your naysayers, there's still going to be people who have just not really been exposed to testing and quality as a concept beyond unit testing. But I think there's that aspect. And then I think, yeah, with just APIs is just the explosion of APIs as an architecture. You know, SaaS, microservices, IoT as well. Like, in my mind, I just treat IoT devices as just physical web services instead of, you know, they communicate in HTTP, but they can pick. They communicate in physical language as well. So I think just as more and more architecture moves towards that sort of approach, how you think about your testing, how you think about risk, how you understand the product that exists in front of you, changes from this sort of the monolithic to more of these distributed systems. And that's not to say one is better than the other. It's just I think that's the way the way the industry is going in general. So that's why I think there's more of that demand.

Nice. Now, the book is going to be broken up into three main sections, I believe, and unless you're going to add more, first one is the value of API testing. Second part is going to be beginning our test strategy. And the third part is expanding our test test strategy. So I like to dive into each one really quick to give people a flavor of each chapter, why they need to grab this book and read it. And as you go along, you'll be adding to it. But anyway, first part is the value of web API testing. Ob question is, why API testing. You touched on a few of this already, a few reasons why someone may want to be interested in web API tests. But are there any others?

Yeah, again, it's sort of. It's sort of talking about testing and quality mindset in general, as well as a conversation I keep having with, with my editor. Is this sort of, you know, am I talking about APIs or am I talking about testing? It's difficult sometimes to not slip into the sort of the bigger picture. But, yeah, so it's starting off with just a case study of my own experiences of working on large API platforms and sort of not what I've learned in terms of the things I've done, but the challenges that are faced and how they kind of relate a lot to just challenges everyone faces in terms of building quality products and why we need testing in there. And then there's a lot I've borrowed heavily from James Lindsay's white paper on exploratory testing strategy and sort of kind of expanded that more to like a wider thinking about testing. But then, yeah, like it's laying the groundwork, so it's laying the mindset. So why do we need testing? Why we need to think about quality? Why don't we need to understand that quality is in the eyes of the users, you know, that sort of idea and how risk impacts that. And then another aspect as well, which I see, it's an interesting thing. I see a lot of, in a lot of specialist training so in security and infrastructure, in automation and testing is the idea of modeling, the idea of just building a sense of the context that you're in. So there's a whole chapter on like just getting to grips with sandbox API that we're going to, that's been used with a book. So it's, yeah, getting comfortable with that, understanding the context of that system. Because I think if you don't really understand your context, you don't really understand like you know, what tools are being used, what libraries are being used, how many APIs are there, how do they all connect together, how are they transporting information across each other, who's developing these things? I think all those questions need to be at least asked if you might not have an answer. But they need to be asked to help establish your understanding of like the situation you're in. From there your strategy can begin rather than guilty as charged, sometimes going in all guns blazing, going, we're going to do everything like this because it worked last time and it's not how it works.

Nice. So how experience do you expect someone to have when they pick up this book, starting from chapter one all the way to the chapter 12 so far?

Well, yeah, my attitude is it is going to be someone who has some experience of working just in the industry and maybe some experience of working with APIs. My attitude is that the reader is coming in because either they've joined an existing project or team, they might have some understanding of different activities. So someone might be quite comfortable with exploratory testing an API, but they don't really know about testing API design or, or the automation side or it might be equally flip round. So I think some sort of experience is going to be there, dropped in or a new project that's going to be building APIs, that sort of idea. So some comfort with HTTP as well. There was a really good point that was made when we first started talking about the structure of the book with the publisher and they said because I'd done a whole chapter which was detailing every part of HTTP and they were like, well, they don't need to read this, they could Just go and look at the RFCs. And whilst that might be a little dry, it is true, you know, all of that information exists for free. So it's what are you going to take that goes beyond that? So, yeah, some experience, I think, but not, not a whole lot, but just enough that you know that you can at least identify the problem that you're facing and then hopefully the book will have some of the solutions. Who knows?

Nice. So is this book game preaching at the choir like testers or is it also developers? I like how you said it's infused with quality and testing in general. So are you trying to aim a bigger audience than just, you know, die hard testers is also developers as well?

Yeah, yeah, absolutely. The way that I structured part two and part three is that I think everyone needs to read the first part because like I said, that's the fundamental mindset that you need to have to succeed with testing in general. It doesn't really matter what you do, but as long as you start identifying what quality means to someone and trying to make an effort to improve that rather than just confirming correctedness of requirements, I think if you've got that mindset, yeah, it doesn't really matter what your role is. And then the actual chapters after that, you could be like, well, I know that I want to do a strategy which involves testing in production and performance and automation because we don't have anyone with exploratory testing skills. Or I might be drawn to the exploratory testing, the shift left testing, API design sort of place. So yeah, I'm trying to strike a balance of not being too introductory, but also I think not just showing but explaining the reasons why and the value behind the things that. That are done. Because like going back to the testers, I think a lot of testers are very good at doing testing, but sometimes they're not necessarily very good at explaining why they're doing something or what the value is. So there's stuff to be taken from the book which is not just the how do you implement automation, but like, how do you. How do you talk about automation in a way that means that you're going to get value from it? How are you going to convince your team that exploratory testing is a worthwhile approach? That sort of idea as well.

Nice. So in part two, as you mentioned, it's beginning our test strategy dive into testing API Designs, exploratory testing, APIs, functional web API automation and formulating our test strategy. You spoke at one of my conferences on, on the Topic of Selenium with API and it was really a. It got a lot of feedback, a lot of people really enjoyed it. So do you cover anything with Selenium and how you can leverage APIs to help you with Selenium? Is that way out of scope? This is just, just API testing?

It is, yeah. I've kept the UI aspect out of this because I think that to bring that in, I think we have to start talking about a lot more about just UI testing in general. Like how do we unit test, how we do visual testing. We need to think about accessibility testing, all of those sort of bits and then yeah, the idea of leveraging APIs and stuff, stuff for that way. I don't think that I will talk very much about Selenium or UI automation in general, but I certainly will talk about how you can leverage APIs in your exploratory testing and how actually you could build these tools. So going back to those talks that I've given before is part of it is just identifying the opportunity. So Richard Bradshaw, boss, boss, super friend to relate back to the old podcast, he talked about the Jigsaw principle. So I think if you understand you can use APIs to generate data very quickly. Think it takes much to go, oh, hang on. Actually I could use this elsewhere if I have like a UI component to my work as well and there'll be a lot of discussion around mocking as well. And I think that that's kind of the same sort of principles apply there.

Great. I think a lot of people hate me in the community because this, but I can't help it. I love tooling, I love tools, I like talking about tools. So what tools are mentioned here? Is it tool specific, is it more high level? And then they can choose whatever tool they want to use to implement it. Do you have any recommendations that you're going to go over?

I've picked some specific tools because I wanted concrete examp and I wanted people to be able to do things. So at the moment, the tools that I've covered what actually exists in the book so far. So on the automation side it's using Rest Assured. But the way that I use Rest Assured is I don't tend to rely on the given when then syntax within Rest Assured. So I like to abstract all of that. So arguably you could build the framework that exists in this book. You could replace Rest Assured with Spring mvc or if you were using a different language, you could have Supertest or HTTP client, you know, Rest HTTP for Ruby. I deliberately chose Java for my language as well for the book, predominantly because I was like, well, if this is API focused, what are the dominant languages that are used for APIs? And it was Java or C in my mind or Net, and my. Net is quite rusty so with a Java sort of approach. But I do try and talk about things in. So whilst there's explicit implementation and exercises that you can follow, I do talk about the theory and the mindset behind it. So you can do things in your own language that are kind of arranged in the same way. And then there's things like using browser dev tools. So even though I'm not going to talk about UI stuff, you know, DevTools is fantastic for learning about APIs if they're driven by UI. There's a little bit of Wireshark in there, which was fun getting that set up the first time. And I'm writing the Testing in Production chapter as we speak and I've been playing with Honeycomb and I must admit, testing in production is probably one of the weaker areas of my own personal experience and I've been getting a lot of support from Abby Banks on that. But I've suddenly become a convert and stuff. Like the Honeycomb stuff's awesome. SRE stuff really, really interesting. So much like that's an interesting thing where developers are very good at that, but testers don't realize that can be a real asset to our strategies. So lots of tools and there'll be more to come.

Yeah, I love it. I love how you mentioned SRE testing and production, because what really made me smile about this book was the last section we actually go over performance testing, security testing, testing and production. I'm a big believer that testers should involved in these activities because they're the best people probably to do or know or help guide almost or shepherd people in these activities. So, you know, why did you feel the need to cover performance and security testing? Because sometimes people say, oh, that's outside the realm of my functional automation that I usually maybe focus in on.

Yeah, sure. So that's because I fundamentally disagree that a test strategy is just purely on function. Like the driver is to start thinking about holistic strategies, thinking about the different risks and the fact that these different activities address those different types of risks. I think what's one thing I'm mindful of, and I'll be curious to know what people think when those chapters come out, is that though they're in the third part, by no means am I saying that they're a specialism, they are an equal part to A test strategy as a whole, they just require different thinking. And the security ones, the one I'm probably the most scared of at the moment, because that's my weakest realm as well. But it's like the idea is kind of like what you said. It's not, here's some security testing tools to use. It's more about, okay, we talked about integration, part two, about just establishing a strategy. Now, how can we take those activities and just put a security lens on it? How can we take those activities and take a performance lens on it? That sort of idea. So less about specialism, but more I like, like you said, like the idea of expanding your sort of your world view of the type of risks and stuff that you've got to mitigate and consider.

Love it. Love it. I'm really, really happy you're going to really get that message out. And this book hopefully is going to help really lead the way. So another thing is people may not be familiar with Manning. They have this MEEP program, I guess it early access. You mentioned you written two chapters. So do you get feedback during those two chapters? And if so, what is the feedback you've gotten so far?

I do, yes. So it's people who are like, familiar with like Lean Pub. It's the same sort of mindset and it's really cool to see, like a publisher embrace it. So, yeah, the chapters exist. If you have purchased the book through early access, then you can basically highlight a section and click a comment. I think you get to choose between two things, which is either I'd like to start a discussion with the author about it, or errata, so you spelt this term wrong or stuff like that. Manning are always pushing people to kind of not worry too much about the grammar and the spelling. So as long as it's, you know, a vaguely comprehensible sentence, then it's okay. But the copy editing all happens at the end. But yeah, so what I get is I get people highlighting sections and dropping questions in. Or like, I don't understand what this means. Or I tried this, but I struggled with that. So, yeah, I'm getting direct feedback and then I can comment back on that. I've just had my first round of reviews come through, so it's interesting. They do it in two ways. So they have early access. So that's paying people from the public giving feedback, but then they also select a panel of people and they go through the same process. But it all comes back to me in one, one big sort of feedback document. So I now have to go through, look at those discussions, respond to them, have conversations, make those tweaks and stuff. And then once we feel happy, then another version of that chapter will come out and it will just keep going until we get to the point of publishing. So there's a real opportunity for people to actually help me forge the path and ensure that I'm communicating things in a clear manner as well. Which again, that's something that we administrative testing have seen and gone. That's really cool. We should do more of that because a lot, of course, making can feel very like you're just in a room recording and stuff, whereas if you're. As soon as you're producing something, you're getting some sort of feedback. I think that's super valuable and it's really helpful.

Absolutely. It was really awesome to see this. One of the few times I've ever went to Twitter and after I left Twitter, I felt good about myself was seeing this, this book. Because this book I think is going to be awesome and it really, really got me excited. But Mark, before we go, is there one piece of actual advice you could give to someone to help them with their automation API testing efforts and what's the best way to find contact you or get our hands on the early access of your book? Testing web APIs from Manning Publishing.

Sure. So I'm going to steal one of our automation and testing principles. And the best bit of advice I can give for people in automation space is problems before tools. You know, spend time understanding your context, spend time trying to understand what it is, what the problem space is, what it is that you're trying to solve. Then think about the tool that's right for you. Because the people who are most successful in automation aren't the ones that know all the tools. They're the ones who know how to set out what the problem is and then select the right tool. Because choosing the wrong tool doesn't mean things go wrong, it just means things take a whole lot longer. And if we are expected to deliver quickly, we can't be dilly dallying, to coin a British phrase, in terms of getting in touch. I'm on Twitter 2bittester with the number two. The book is available on Manning.com testing web APIs. You can just search for it or check the MEEP catalog and it'll be there. And I'm always sharing links as well about it. And also it's been really interesting actually as well. Like I'm on LinkedIn a lot more now because the book's getting a lot more traction in that space than in Twitter. So it's making me learn things about like about social media as well, which is quite cool. And then obviously Ministry of Testing markeryoftesting.com if you want to ever talk about MOTT stuff.

Thanks again for your automation awesomeness. The links to everything of value we covered in this episode. Head on over to testguild.com a355 and while you're there, make sure to click on the Try for Free Today link under the exclusive Sponsor section to learn all about Sauce Labs awesome products and services. And if the show has helped you in any way, why not rate it and review it in itunes? Reviews really help in the rankings of the show and I read each and every one of them. So that's it for this episode of the Test Guild Automation Podcast. I'm Joe. My mission is to help you succeed with creating end to end full stack automation awesomeness. As always, test everything and keep the good. Cheers.

Thanks for listening to the Test Guild Automation Podcast. Head on over to testguild.com for full show notes, amazing blog articles and online testing conferences. Don't forget to subscribe to the Guild to continue your testing journey.