The Complete Software Tester with Kristin Jackvony

Welcome to the Test Guild Automation Podcast where we all get together to learn more about automation and software testing with your host, Joe Calantonio.

Hey, it's Joe. And welcome to another episode of the Test Guild Automation Podcast. Today we'll be talking with Kristen Giacovone all about her new book, the Complete and I mean the complete Software Tester Book. This book is 500, I think 40 pages. I think I counted the chapters, like 124 chapters. So it is jam packed with awesomeness if you don't know. Kristen is a principal engineer for Quality at Paylocity. In addition to her role at Paylocity, she is also has a blog called Think Like a Tester and is a frequent speaker at many testing events. She actually spoke at Automation Guild this year and she really brought the good stuff as she always does. So you don't want to miss this episode. Check it out.

The Test Guild Automation Podcast is sponsored by the fantastic folks at Source Labs, the cloud based test platform that helps ensure you can develop with confidence at every step from code to deployment for Every framework, browser, OS, mobile device and API. Get a free trial. Visit testguild.comSourceLabs and click on the exclusive sponsor section to try it for free today. Check it out.

Hey Kristen, welcome to the Guild.

Hey Joe, thanks for having me back.

Awesome to have you back. You always bring a lot of awesome stuff. Is there anything I missed in your bio that you want the Guild to know more about that I may have forgot about?

I don't think so.

I think you've covered the basics, where I work, what my role is, and the fact that I'm a frequent blogger and that I speak at conferences, but the big thing, of course is my book.

Awesome. Absolutely. And you know, I know it's kind of cheesy and I tell authors this, but the first question I always like to ask authors is why? Why? Why did you write? It's not an easy task. From my experience, it's a monster of a book. Why write the Complete Software Test Tester?

Yeah.

Yeah. Well, I was really driven to write this book by my experience when I first became a software tester. So you may have heard before. I can't remember if I've brought it up in previous conversations, but I came to software testing very late in life. I had already had another career as a music educator. And when I discovered software testing, and this was back in like 2009, I wanted to learn everything I could about software testing. And I found that there weren't really a whole lot of resources at that time. Now, of course, since then, it's improved a lot. We have especially, like, lots of great resources on the web, lots of courses, things like that. And there are some really great books out there. But there wasn't any book that was like really complete where somebody who wanted to get into software testing could like, get this big tome and say, okay, what do I need to know? So that was definitely one of my intended audiences. I really wanted to help them out, but I also really wanted to help out testers who are manual testers who want to get into automation. And they might be a little lost at first. So I really try and include things in the book that sort of help people understand the basics so that then they can use those basics to go out and do those web searches or take those courses to like, learn more.

Of what they don't know, you know.

And then I also wanted to have automation engineers learn more about the basics of software testing, good software testing, and test planning. Because sometimes automation engineers might come into that path from being a developer and they might not have really done a whole lot of thinking about what really am I supposed to be testing here or what's a good test strategy. And then finally, I also wanted to help developers who wanted to understand testing better. We're moving towards a model of having a whole team really own quality. And so when developers understand what to test and why to test and how to test, that makes the whole team work better. So it was really for those four audiences, and it was a lot of work, but it was a labor of love.

Awesome. So, Kristen, as you mentioned, I like how you wrote it from someone that's never done testing all the way to someone that is probably a very skilled automation engineer. So I thought we'd start from the beginning. Part one is why do we test? When you meet someone that's new to testing, a lot of times they may just think testing is automation nowadays. So how do you define what is testing? I know you covered it in the book, but give people a little, A little something, a little meat for them to go, wow, this sizzles. I need to get this whole. This whole book.

Well, I'm not sure if this exact definition is in the book, but. But basically the reason why we test is because we want to get information about the product. We want to make sure that everything that we're expecting is going to work a certain way is indeed, in fact working that way. And we want to make sure that the product works under a whole bunch of different scenarios and that's always going to be important. And while we can have a lot of test automation around all of those kinds of things, there's also, there's no substitute for the human brain. And it's the human brain that is thinking about, well, what would happen if I did this? What would happen if I combined these two features? What would happen if I hit this button 75 times? So that's why we will always need testers. Even if we're on a team where we've got a lot of great developers who are writing a lot of our automation, it's up to us testers to really think about what kinds of things can go wrong. How would we want to test for that?

So besides, you know, why do we test? Your second chapter is on manual testing. How important do you think, once again, new folks entering the field, manual testing is? Is it still seen as a critical skill or do most people overlook it?

I think a lot of companies overlook it.

And I will actually give you an.

Anecdote from my own company from when we hire software testers, one of the things that we do, one of the very first things we do is our candidates to find some bugs for us. We give them the URL of a buggy application and we say, take a look, play around, do a little manual testing and find some bugs for us. And you would be astounded at the lack of bugs that we get sometimes from candidates. And sometimes I go back and I look at their resume and I see they've been a software tester for 13 years, but they can't find the bugs. So it's still, it's very important because that's really, that's why we're here. We're here to make sure that our software is going to work like it's supposed to. You can be a really amazing automation engineer that writes all kinds of great code, but if you're not testing the right stuff, then you're still going to wind up with buggy software that no one wants to use.

So are there any manual testing techniques you think if people just knew about would make them a better tester to find those bugs that maybe the applicants are missing? I know you have chapters on testing, postal codes, data fields, phone numbers. You have a whole bunch of stuff in this section. So just curious to know which one you think really is one that, you know, a tester could latch onto to really get a lot of benefit.

Here's one that's really, really helpful. And this one has actually tripped me up more than once. And that is testing spaces. So, for example, let's say you're doing an upload of a file and your file has spaces in it, and then for some reason that file isn't processed correctly. Those are the kinds of things that can tend to trip people up because they might have a test file that they use over and over again that doesn't have spaces in it. And then also every time you have a text field trying to pass in an empty string instead of passing in something. And I think the reason why a lot of us forget about that one critical thing to test is because you can't see it. So when you can't see it, you don't really think about testing for it.

No, that's a great tip. I used to always have issues with trailing characters. Sometimes I know the text would get corrupted and you couldn't see it. Like you said in like, what's going on? You have to put the text and do a backspace to get rid of the delete and it would work. So that's a really great tip. Cool. So besides manual testing you, that's part two. Like I said, there's a whole bunch of chapters in between. You get into how applications work. So in this section, is there something you think fundamentally people need to know about. Everyone needs to know about applications and how they work to be successful at testing.

I think one really important thing, especially in this day and age, is to understand the difference between encoding and encryption. So encoding is what happens when, for example, if you've got a URL and there are spaces in your URL, you've probably seen like, oh, there's a percent sign and then a number 20. And so that sort of encodes that space in the URL, whereas encryption is actually encrypting the values with some kind of like a hash or other kind of mechanism that really makes it impossible to decode unless you are the actual person who is supposed to have access to that information. And so one common security problem when, especially with, like new developers, when they first start making an application and they want to put in username and password and that kind of thing, is that if you just do like base 64 encoding of those values, they're not secure at all. You can take those encoded values and you can just find any simple base 64 decoder on the web and decode and see the password. And I actually have that as an example in my book. I've got like, see if you can get my password. And here it is.

Very nice. So you just mentioned something that Triggered a memory here I spoke to, I don't know, like a few months ago, I think API testing and I think you mentioned something about writing an application and I thought it had something to do with this book. So when you were talking about encoding and everything, is that the application you wrote to help test against for this particular chapter, this section, I did make.

An application for use with this book. Fortunately that basic level encoding is not how my application is encoded. I used JWTS to encode the username and password, so it's a little more secure. But yeah, I wanted to have an application that people reading the book could use to try things out. And I especially wanted to have the UI and the API be connected to the same application so that somebody, for example, who's learning about API testing, they could maybe do a post and add a new contact to that database and then they could go to the UI on the web and they could see did it really get added. So that's why I created that. And so there are a number of my chapters that have hands on activities that use that application.

Great. So we're still in how applications work section and as I mentioned it, this book feels like it would benefit both the beginner and an expert. And the reason why I say I'm older, so if I was to look in how an application works, I would assume, oh, I know how an application works. But then at the end of some of these chapters you have like testing with the non rational database serverless architecture. Two things may not have a lot of experience with. Would that benefit someone that is like more of a experienced tester?

Definitely, yeah. Really? My, my whole point in touching in a basic way on a lot of different topics is that I want to give everybody just a little bit of exposure so that then that's kind of like a hook that they can use to get more information moving forward. So a perfect example of that would be non relational databases. Like to somebody who's been dealing in SQL their whole career, non relational databases seem really, really scary. But, but they're actually not that scary. It's just they just store information in a different way. And so I tried to give an example. I've got a chapter about or a couple chapters about traditional relational databases and how they work. And then I've got a chapter about a non relational database that uses some of the same kind of information. So you can see how they really differ.

Absolutely. And so we get into part four. I don't know if we'll be able to cover all 10, but part four is API testing, which I think is your bread and butter. I guess when people approach a book like this, sometimes they worry like, oh, I don't have the tooling. So before we get into API, or even if it applies to API testing, does someone need to have particular Selenium or does it matter? Are you just using it as examples? How does that work?

Everything that I am using is free, so even there are some cases. For example, Postman, I'm using Postman. Postman is free. Yes, there are paid versions of Postman, but you don't need the paid version of Postman in order to do the exercises in the book. And then another example would be Applitools. I'm using applatools for some in my chapter on visual testing. And while applitools is also a paid product, you can, you know, if you're just like an individual learner, you can have a free account that you can use to try these things out. And another thing that I've done is I have in my GitHub repo, I have a whole bunch of different code samples that people can use, pull down, take a look at, and everything in there is completely free.

So before we skip API testing, like I said, I think it's one of your bread and butter topics. Is there anything you think most people misunderstand about API testing? Word that correctly.

The biggest thing is that people aren't using API testing enough. They're going to that UI testing, like, too much and too often. And I think the reason why is because I think a lot of us testers who get into test automation, we may have come from not from a coding background, but from, you know, some other career. And we get into testing and we really like it, and then we say, oh, we want to start doing automation. So we think about automating things in the same way that we think about how we're manually testing. So we're, you know, poking around, we're clicking buttons and those kinds of things. But so much testing can be done below that UI layer. And API testing is a really great way to do that. It's a really, really great way to test lots of application rules, you know, complicated scenarios. And it, you know, as you saw in my Test Guild presentation, it can cut your testing time in half if you're like, setting up your tests using API calls instead of like going in and like, you know, clicking add new contact and manually adding that in every single time. So, yeah, that's the biggest thing.

Absolutely love it. So I think another People struggle with is mobile testing, which I think you cover in the next part. 5. What are maybe some challenges you see with mobile testing that you think testers need to know about and may struggle with? You have a whole chapter on it, but maybe one or two points you think someone can take away from this?

Yeah, sure thing. So I think the first thing is operating system. I think everybody understands, yes, there are differences between Android and iOS, but when you're testing mobile, you definitely want to make sure that you are testing on both of those because you will see bugs in one that you don't see in the other. So that's a very important one. Also network. And that's something that people don't think about when they're testing the web, you know, because the web is so solidified now. I mean every now and then you might get somebody who has a slow request or something like that. But to think about like what is your network provider that can sometimes. I found a bug once, this was several years ago, but I found found a bug once where the location services were working completely different if I was on Verizon versus how it was working on att. So the bug was only in replace. So that's another one. And then of course screen size. Oh, and then finally also landscape, portrait and landscape. You got to remember that your users might want to view your application in landscape. So you definitely got to make that twist to do your testing.

Great. So like I said, this is the complete software testers guide. And I was really excited to see the next three chapters you cover. Performance tests and accessibility testing, Security testing. I think these areas a lot of traditionally testers may not have really focused in on. Why include what people call, I know some people hate this term non functional type topics.

Well, I think because I think we're moving to a time where we really need testers to be fair. Full stack testers. And so testers need to understand about what the common security holes are and they need to understand how to do at least very basic performance tests. There are some companies that have a dedicated team to performance testing or a dedicated team to security testing. And that is really awesome. But there can still sometimes be things that fall through the cracks. One of the things that people don't often realize when they think about security testing, they go right to, oh, this must be really, really complicated. I must need to make this complicated SQL injection here and put it in. But one very, very basic thing for security testing is just making sure that if you're not an admin that you can't see the thing that the admin is supposed to see. So to think about those kinds of things and just basic like how fast is your application responding? Those are things that help you be a really, really valuable great.

And how about performance testing? I think of performance as I think of a stress test, but nowadays it seems like a lot of applications really rely on almost like a thick client front end browser. When you cover performance testing, what type of topics are you talking or covering?

Well, one of the things I talk about is just thinking about exactly what kind of load you're going to be expecting and when. And so you know one example that I give, I say let's say you've got like a work for a store that's like a web store that sells chocolates. And so you know that February 14th, that's going to be like your big hard hit day. And you think about how many orders might you get that day? You know, how many orders did you get last year, how many orders maybe did you get at Christmas? And you can try to make a guess of well, how many orders do we think we need to be able to process and then make some decisions about what to test from there? I also make a distinction between really putting load on and just measuring response times. I mean even just measuring how long it takes your page to load can be very valuable because if user goes to your webpage for the first time and they have to wait five seconds to see everything, they're probably not going to come back. So it doesn't always have to be let's break the application with our heavy load. It can also be let's think about what kind of load we want to support and let's just take a look at what the typical user experience is going to be.

So besides chapters, parts of security testing, performance testing, accessibility testing, you also touch on software development basics. So do you think that to be a successful tester you need to know some sort of coding or what do you believe the skills are that a tester needs for software development?

I do really think that good software testers need to be coders. They need to understand how to work with code, they need to understand how to write clean code. I think maybe a decade ago that wasn't quite as important. But in today's agile environment, where new features are getting pushed out every week or every two weeks, it's very, very important to be able to get that testing done quickly. And if you can't write that code to automate your testing, you're going to get left in the dust, you're going to be the bottleneck that everybody resents because you can't create that automation. So that's why it's really important. And what I've tried to do in the software development basics is to take some things that software developers and automation engineers take for granted and explain them sort of from the very beginning. So like for example, a new manual tester who was maybe coming into a team and said, oh, you know, I'd like to learn some test automation now. And then somebody says something to them like, okay, so go to Git and then pull down this repo and then CD to that folder, make the code change and do a pr. Like that manual tester is going to be like, what, what, what does that even mean? You just, you just use like seven words that I did not understand. So I really wanted to include chapters about. This is what git is. This is how you use git. This is how you work with the command line. Here let's practice like doing a CD to get to a different directory. Here's how you would make a file. So I wanted to go back to those real basic so that people who are missing those things or maybe didn't quite understand or people who are like completely brand new could have enough information to get started with.

Great. And so finally, in part 10, we finally hit automated testing. So yay. So.

So people need to buy this book because we don't have time to go over it. So let's just give them a little sum from automated testing, which, which caught my attention is the automation wheel moving away from the testing period to the test automation wheel. So let's give people just a taste of what is the test automation wheel. So they have to get this book to know really more in detail of what the heck it really is.

Sure thing. So I mean, everybody's heard of the pyramid, everybody understands, you know, oh, you want to have like mostly unit tests and then you want to have those services API tests and then you want to have the fewest UI tests as possible, because that's going to be. Those are the most, you know, to have the pyramid that way is most reliable and quicker and all of that. But I also really wanted to think about automated testing in terms of what you're actually testing. And so I created this little thing called the automation test wheel. And so it touches all of these different areas that you will want to automate. So it starts out with your usual unit tests, then it moves to like component tests, like can you hit a database or Whatever. Then it goes to services tests, which are your API tests, then your typical UI tests. So you know, most of the stuff we've covered already. But then it goes on and it talks about visual tests. Like, you want to automate your visual tests so you don't always have to have like a manual tester's eyeballs on that one particular page. And then automate your security tests. And this is really useful to catch like, small security holes that a developer might introduce that otherwise you might not notice. And performance tests, you can automate your performance tests. And then I feel like there's one more. Well, there's eight. So if I didn't get to eight, then you can get my book. And you can see the last one was.

Absolutely highly recommend you do. So, you know, once we get over automated testing, we get to testing strategy. There's a lot of stuff you just went over, and it's a lot of things to do for a tester. How do they prioritize all this in a testing strategy? Do you go over maybe what people hit in a sprint or what is the testing strategy section cover?

Well, one of the things I talk about is smoke tests and how to decide what to put in a smoke test. And I actually use an example from early in my testing career. I was working at a company where we had. We didn't have any smoke tests, and we were all manual testers. And we discovered that we had actually missed a feature that broke. And it had broken like a month before. And none of us ever noticed because we had so much other stuff to test. And we were never really doing a smoke test before release. And when we realized that, one of the people on our team created a smoke test. It was manual, but it was a smoke test. It's like, let's check all of these things before we release. And so it was great. We were all using the smoke test, but then gradually, every time a new bug crept in, that particular person was like adding another check for that into the smoke test. So then the smoke test wound up being this thing that took an entire day to run through. And so in my book, I have like a section about deciding what you should put in a smoke test, what belongs there and what doesn't belong there. And then I also have a chapter that I'm really very proud of. I create this imaginary complicated feature, and then I ask people, well, how would you test this? How would you test this very complicated feature? And then I walk them through how I would create the test plan for it, starting from, like, the most basic use case and then sort of gradually folding in additional complexity as it goes on. So I hope that people will find that really helpful.

Very cool. Definitely a must read chapter or a part of this book and you wrap it up finally with software skills for testers. I don't know if this is still the same, but when I was like a tester it was before Agile. I'm gonna get in trouble. But we were always kind of like the misfits or like, I don't know if you ever saw Rudolph the land of where they had the misfit toys. That's what we all felt like. And we really have poor skills. Mostly introverts, really weirdos. So nowadays I don't know if it's like that. So what soft skills you think testers need? Maybe that you see, maybe I guess you can't lump everyone together, but you've seen in your career that maybe can help them with their career.

Well, one of the chapters that's a lot of fun is Testing Personas. I talk about some testing Personas to avoid. So one of them is Persona of somebody that finds an extremely obscure bug and then can't reproduce it and then spends like an entire week trying to find it again while meanwhile, you know, the release has happened. Everybody's trying to move on to something else. But then at the other extreme would be the person who just like runs the test and doesn't think about like going outside the box. So there's a chapter about that. So sort of how to avoid being that testing Persona that might not be the best. And then another chapter I have is about how to work with developers to make sure that, you know, you're not that person when you walk over to their desk, either, you know, virtually or metaphorically, they're not like, oh no, she's back to tell me about another bug. You know, making sure that you're really working collaboratively with them. So I think that's really helpful as well. And then there's also a little bit about staying organized in time management too.

Great. Okay, Kristen, this has been jam packed with actual advice. I usually ask everyone for one piece of actionable advice, but I think you've gave like at least 12 pieces of actual advice. So I guess the question is how can someone get ahold of you or get their hands on your new book and where is it located and what's the best way to find or contact you?

Yeah, okay. Well, you can get my book on Amazon. It is a Kindle book at the moment. It's only an ebook. A common misconception is people think that they have to own a Kindle in order to get a Kindle book. You do not have to own a Kindle. You get the Kindle app. I don't own a Kindle, but I've got the Kindle app on my phone, on my laptop, on my tablet. So all you need to do is just get that app, you buy the book, and then it just downloads to your Kindle. So to your Kindle app. So you can definitely get the book that way. I'd also like to point out that if you have a Kindle Unlimited account, the book is free. You can read it for free. And then to find me, you can find me on LinkedIn, you can find me on my website, which is thinking tester.com those are probably the best places.

And currently the book is only seven bucks people. So get your hands on this book and make sure once you get the book to email Kristen, tell her how much you enjoyed it and give us some feedback.

Write a review.

Write a review. Oh yeah, absolutely. Write a review on Amazon. Easy to do. Cool. Awesome. Thank you Kristen for your awesomeness. Really appreciate it.

Thanks.

Thanks again for your automation awesomeness. The links to everything of value we covered in this episode. Head on over to testguild.com a390 and while you're there, make sure to click on the Try for Free Today link under the exclusive Sponsor section to learn all about Sauce Lab's awesome products and services. And if the show has helped you in any way, why not rate it and review it in itunes? Reviews really help in the rankings of the show and I read each and every one of them. So that's it for this episode of the Test Guild Automation Podcast. I'm Joe. My mission is to help you succeed with creating end to end full Stack Automation awesomeness. As always, test everything and keep the good. Cheers.

Thanks for listening to the Test Guild Automation Podcast. Head on over to testguild.com for full show notes, amazing blog articles and online testing conferences. Don't forget to subscribe to the Guild to continue your testing journey.