Why Security Testing is an important skill for a QEs with Boris Arapovic

In a land of testers far and widely journey seeking answers, seeking skills, seeking a better way O the test Illumination Testing Podcast Guiding testers with our nation awesomeness from ancient realms to modern days they lead the way. Oh, the testicle of nature Testing podcast.

Hey, join me today as we dive into the challenge of proving why I think and Boris think security testing has to be a top priority for quality engineers and Software testers in 2024 and beyond. Today we'll be talking all about it with Boris on why security testing is an important skill for QEs. So if you don't know, Boris is a senior quality engineer with a passion for for agile testing, test automation and one of those rare unicorns, cybersecurity. Currently he works as a director of Quality Assurance at IBM ix so he really knows his stuff. I'm really personally excited about this topic. You don't want to miss it. Check it out. Hey Boris, welcome to the guild.

Thanks a lot Joe. Really nice to be here. I am listening this podcast more than one year even I think more than two years and it's really excited now that I'm talking and not just listening.

Yeah, I'm so happy you actually reached out to me. So I just give a shout out to that. Anyone that has a topic you want to learn about, you can always reach out to me on LinkedIn and this is one I'm really been not struggling with but you know, kind of stumbling with. For the past few years I used to have a separate online conference called Secure Guild and I can never really get it off the ground because I always get pushback from testers saying hey, it's not my job security testing. I have my own team so I guess that's setting the background. Before we get into that though, what got you into cybersecurity or security testing?

So in general data set, I mean I fully understand the concerns for example quiz have in terms of, you know, I'm not a cybersecurity engineer and that's also not what I'm saying. So the depth to know how that you need is really like it's huge and it's really hard to get if you don't focus on this fully. However, what I think is that kind of cybersecurity fundamentals will help the application itself that we deliver to the client and basically also as me as a queer when I gain this know how I will also be able to provide additional quick feedback before it may be some things are shipped or maybe going to a cybersecurity company or even a Cybersecurity engineer because at the end of the day it's about the customer satisfaction who uses this application. Right. And we have to work and cope with it to avoid maybe some data being leaked or in general out of my experience this actually not even super highly sophisticated security hacks, it's sometimes super simple that every query can do and where we can really add a lot of value in there.

No, it's a great point. I've seen SQL injection has a problem since I started my career over 25 years ago and if you go to a, you know, OWASP and things like that, it's always still a top security concern. So a QA then as you mentioned, doesn't need to necessarily be a security expert. But how do they get that as part of their work then? Or like you know, that they're doing a sprint planning the definition of done. They need to do a whole bunch of activities. How do they know or how do they get their teams to be aware that yeah, yeah, we're not necessarily the security team, but we could still do test cases that help with the security of the application as we're building it.

Yeah, correct. So I think there are many ways on how we squeeze can contribute. I think first of all it actually starts in having some static application security testing place that actually helps also the developers and when we develop code for example, just to also detect already some security issues which might be having hard coded password as a prime example. Trust me, it still happens. It's not that this is something like no, this won't happen on application, this for sure can happen but I think it's as a first step we as always can help to set those things up, maybe even consult involve a cybersecurity engineer improperly to set this up properly. As I said, for example checking the source code, but it also might be checking the packages that you have and you know about Node js, you typically have the NPM and when you just, you know, type NPM install it will print you out some security availabilities that you will have on packages. Right. And I would say we sqa also should drive this together with the developer, make the developers aware of it and also kind of, I wouldn't say fight for that but make also time that we can also fix such issues because those are typically low hanging fruits when it comes to SAST as a next step typically because you mentioned agile and sprints and so on. I think what we can do is as you said already, create some test cases where we also cover Some of the edge cases, let's say for example we implement the login feature. So when I think about the login feature my first thought would be what about the recapture do we need? Like how do we protect it? This is like I don't need a speed to be necessarily a cybersecurity expert to bring such perspective. And I think we as always quite often have the perspective, the technical but also on the other hand the business perspective. And this is like also think about on I as a user how I can be protected in a way from it's really quite often really obvious security issues. And I would say that's of course the test cases, the SAS that I mentioned. And I would say also where we can bring a lot of value is in designing session based exploratories, test sessions together with the teams. So you know, we have this, those for example Personas that I can do and imagine now you create the Persona where you have the hacker. How is hacker called? Where you can see this is the objective of the hacker, this is the approach of the hacker, this is what the hacker want to do. And then you organize such a session where have like let's say also typical Personas. But you have also like the hacker Persona. And you then just let for example the developer pick the Persona and really think about they are now the hacker. What could they do? You could then for example do on the release level, but you could also do that on a feature level. What I just mentioned, login for example. So how could such a hacker maybe right. Bypass the authentication mechanism or what could a hacker do? And yeah, as I mentioned. So for exploratory testing we have one path is the Persona and the other path what we also as a quantities could do is create charters where we could say, you know, for example we could create in case of the login. As a hacker I want to be able. I want to bypass the security restriction by providing some SQL injections or we could make it broader by some injections and they are like different, you know, things what you can use there to apply it and you can be really creative. And that out of my experience that's typically what the developers like the most to test. It's not having the checklist but it's really like, you know, having this creativity that they can play out and that really helps also to boost it. Yeah. So and additional point what I want to add there is of course we as quiz test. That's also like when we test a feature we can bring in this perspective of course But I think also at moment when we have like very critical releases, very critical features that we want to release. Let's say for example we add a new payment method within the checkout there. We might then consider a security expert to actually pen test it once we maybe done some basics checks. And I think that's, that's really something that adds a lot of value also from the whole team perspective in addition to that, because sometimes you really need an expert. You can't test all like the, maybe the tricky combinations, but the low hanging fruits. I'm pretty sure confident a scrum team can cover a lot of those things.

All right, so there are a bunch of great points you just made there. The first one that jumped out on me is when I think of testers, I think of them as leaders. And so you just made a great point. They don't necessarily have to do everything, but they do need to be aware enough to say, hey, this may be a task the developer should look at. Maybe I should reach out to the security team to do pen testing like you said. How do you empower your testers though to take that on? Because a lot of times they think like, oh, I'm in a box, I can't step outside of this box. I don't know if you have that problem, but I've seen organizations where a tester is just like they feel like they're in a little box and a little vertical and they can't break out of it.

Yeah, so that's a very valid question I think how you empower them. I think first of all they need some education first to understand it. And typically what we also do, we show some like real examples from our projects to see like really guys, it's really happening and to also on the one hand motivate them. So first to have to know how to, I mean to gain that, it's not easy. But typically also within IBM X, we support people, we have an academy where they can do specific trainings to educate them. So as a first step and once educate, it's then also about a lot about knowledge sharing. So between QAs, even outside of a project, as we are an agency, right. We have like many different QAs working in many different projects, but they can exchange information. And that's also, I would say also how we can, how we motivate our people to also take a look at that aspect because it's really happening on project level in general. So not even talking about our projects that we have, but just I just also recently taken a Look on the website where for example, it was a web shop where I just typed they had like when I do something they contact the backend API and in the backend API just added like a path, a random path. And all of a sudden I was logged in as a user, which I thought like really that easy? It was like anonymous user in a way, but still. So it's really reality and we have to adjust to the reality. I would say absolutely.

So another thing you mentioned which I really liked is the exploratory aspect of it. I used to work on a team where had like eight to 10 sprint teams, but we'd come together every quarter or every four weeks to do exploratory testing and, and that's where we tended to find the biggest and the most most valuable bugs. So can you talk a little bit more about that then how you get the team involved? Because it is quite an initiative to tell your boss, hey, we're going to take these developers off developing and they're going to focus on, you know, testing in a sense.

Yes, that's a very good point, I would say. So in general what we do, we really try. So for exploratory testing, typically you need to prepare very well. Let's call it like that. So at the beginning, before we even start an exercise, we look, okay, are we going to apply Personas? Are we going to do maybe some charters? Create some charters that make sense. Right. And typically we then organize the session, let's say much in advance that there is sufficient time also for developers, you know, to plan things in and everything. And then once we execute it, we are typically very organized. So we have like five minutes introduction where we explain everything and then kind of, they typically use those Personas. We also decide in the team like not to have like, you know, all the people use the same Personas to have a good mix. And then they start and they have their time and as soon as they discover something, they document it. For example, use Confluence and they document it just very briefly. Right. And once, typically it takes one hour, maybe even half an hour. It really depends on the complexity in general, on the requirements. But afterwards we then have a follow up for five to 10 minutes where we quickly discuss the findings. And then typically we as a qua, take this onwards, ask questions directly to developer, if there are some questions, you know, whatever the reported and then we follow up on that. So we don't want in this case we try to keep the work as little as possible for the developers, but kind of we then take on we prepare everything, we like host everything and we also take on the next steps to clarify maybe if something is really a security issue is it maybe a new requirement that we just forgot? Like it could be a recapture on some login registration or something like that.

Nice. So I know in certain verticals like health care and financials, you know you can actually be in big trouble by the FDA or if you're audited if a lot of these things get leaked especially for PII and things like that so and yet once again those testers there sometimes they don't do security and I tend to see that the security team is like completely isolated from everyone else and they just get like verdicts on down from the security team thou shall do that. So I don't know do you have any experience of trying to breaking down that wall? Because there still seems to be testers, developers are now part of the same team but yet we still have you know the security team almost is like this nubius other somewhere else that does it what it does and we have no idea what they're doing. We don't necessarily work with one another.

So I think as of now I can say you like this is like the recipe that works for everything. It always depends on the project and also like how the security team is placed and what their responsibilities. I think for as of now our solution is always to kind of more when we see the need to try to involve some right based on the need but like really to have it in a project permanently it's really impossible. I would say when you take a look at maybe scaled agile frameworks that we have is it safe? I don't know whatever the framework is but let's say you might have literally 10 or more teams working on different features and so on where you can't ask the security engineers for everything from the very beginning to be involved and so on. So it's really I see a lot of companies that don't have like this big security department that you, you know they're typically couple of people and this is kind of the reality that we are facing and that's why I think also why I'm also stating that we also need to gain this cybersecurity know how at least fundamentals because it's really we don't have the people, all the people ready to be able to do that and yeah a hundred percent and.

So my big push always is to try to appeal to someone's self interest and I always tell testers maybe I'm wrong that if you have testing skills that include security testing, performance testing, you're going to be more employable. So how do you motivate your team? Like are there any tools or techniques that you recommend people learn if they want to, like you said, learn the fundamentals. Do you recommend OWASP or any other sites where they can learn more? If like they had this tool on their resume or this technique, they probably would pop a little more than someone else.

I think first of all, actually I was once in a project also together with Quays where we implement with Quays and Developers where we implemented a registration and you know, we implement the registration and remember back then we had some UI tests running with Selenium, you know, to validate the registration. And then on another Sprint we added a recaptcha, Google Recaptcha and I remembered that once we added it, I wasn't able to automate the UI test anymore with Selenium because of the Google recapture. But then I played a bit around with it and discovered well, if I with back then with selenium, with the JavaScript executor, if I removed HTML stuff on the front end, all of a sudden from the recaptcha I was able to complete the registration process. And at first like, I was like yeah, I did it and said like no wait, something seems pretty off here, I shouldn't be able to do that. And I remember that when I was talking with developers, explaining them what I did, like bypassing the front end and so on, I saw in their faces I got some respect from them, right? The first developer said wait man, you're a hacker. Like you know, you see you also get then respect from developers when you show also such things and not just like you know, we are testing from a functional perspective. But I think especially such things, it also is good for your reputation within the team actually.

Totally agree. So you know, you also mentioned earlier static adding static like security scans when developers checking code. Do you have any recommendations for that? Because a lot of times I know developers get. Because just like other tests, sometimes security tests are flaky or they're, they're not really reliable in the sense that you get a ton of them. You don't know where to, where to focus in on. I don't know if you have any experience with rules of thumb that you can do to make those scans really effective.

I would say typically what we do is we do sonarcube. We use sonarqube with the like typical, you have a security rating right in every sonarqube Report. And this is I would say the best thought first to start with that and keep up the rating because this is like kind of, I would say also the low hanging fruit because like every project to have something like a sonar cube in place and you might then go deeper and really having maybe 45 Eric or whatever, but even having already, that helps a lot with the security aspect I would say.

All right, so you are director of Quality Assurance, so I assume you're involved in hiring. Do you look for testers that have security skills when you go through resumes or do you just look for someone that has an open mind of trying different things?

It's really open mind how they approach problems. Because I think the strength of a query is to, you know, the behavior and like the pattern on how you test something. You typically think about, I'm that user maybe and that user, how would that user buy something in a web shop or somewhere else? And I think it's really important then think about, I'm a hacker now and what could I do? Maybe to bypass some security restriction and sometimes it doesn't even need to be like super highly sophisticated, but it really can be more on that level. That's where we hire more for Mindset. However, we need some technical know how, you know, understand like how APIs work. Because at the end of the day we have a lot of web based projects and you know, everything is going API first and so on. And it's then very critical that you understand how API works. You know, what's a get, what's a post and so on. And this is like something we also look for when we hire people that they understand those concepts.

All right, just reminds me of APIs. A lot of times they're not as secure as the front end because you don't see them. So you can't see them. So who's going to mess with them? But yet they seem to be some of the most vulnerable hacks that I've seen over the years. So any advice around making sure you create secure APIs then?

Yes, to be honest, I think it's so my biggest advice is when, let's say as an example you have a newsletter where you can subscribe with your email address and we as tester might take a look at it and say like, okay, I'm taking a look at the front end and there is some regex, right, that validates the email and then I see okay, is it working? Is it not working? But from a security aspect, what's more important is the API itself. Right. So because I as a hacker might just bypass that and then I might provide a very long string, millions of very long strings, which might bring down my database completely. Right. Because it's out of memory or whatever. Right. So I think the biggest takeaway is really on API security, especially on that level. I don't see necessarily, let's say from network security, how everything is configured. Yeah. There you might add also cybersecurity to did it. But, but I think really it's on the level of application security where IQA can bring a lot of value, especially when you take a look at the whole maybe user journey of the customer and understand how everything works together. Right. At the end of the day, a cybersecurity engineer might have like maybe four, five days time to pen test an application and you know, it will always be too little. But theoretically they could like do this for months also.

Absolutely. Boris, this might be. Question. Might be a little odd, maybe it's not odd. So ever since I started in my career as a tester, performance engineer, automation engineer, I've always been told that the job's going away, we're going to be replaced. And now with AI, I'm getting more and more of that. But you have the title of Director of Quality Assurance, which to me is a rarer title because it seems to be more Director of Development now that in charge of testers and qa. So I guess my short question to this is, where do you see if someone's listening their role as a tester? Do you see it as a valid role going forward? Do you see AI being disruptive or replacing testers or anything like that?

I feel back, like feel set back in time again. When Test Automation came out, everybody was talking like, you know, yes, exactly. Why do we need quality engineers or tester or have them want to call us. But I think from my perspective, we should really use this as an opportunity to improve our work as we did it. Right. Test Automation helped us to get rid of, to be honest, a pretty boring regression test that you have to do manually every day. I mean, that's not fun, to be honest. And that's how, let's say Test Automation improved our quality of work. And the fun I have at work, right, it's much more broader. I also have to automate more into development. Right. And I think the AI is also the next big step that will also help us to maybe get rid of maybe additional some things that we probably even not are not that well suited as humans. But there will always be the need for humans to test an application. So I don't see that really happening and I'm pretty sure it won't. It will actually help us to make our job even more fun.

100% agree. What I do see though is people leveraging now AI for security hacks. So is that something you're aware of or something testers need to be aware of? Like how do they keep up to date then that, you know, not only are we dealing with known vulnerabilities, but now we have AI being leveraged to kind of find other vulnerabilities we may not, may not have thought of.

So you mean more now? For example, I use maybe something like chatgpt that is more focused on security and gives me some advice. Or do you mean like more towards AI services? Some like a chatbot or whatever. And how like could this lead to security risks?

Exactly, yeah, both. Let's go with the last, last one probably first.

Yeah, let's go with the last one. So I see this like as a very big need also security is important in that aspect. So there are a lot of vulnerabilities, you could say. But in general they are like two most common. And the one is first the AI jailbreak. Right. Where you can basically bypass the security restriction of let's say ChatGPT. Right. As an example, ChatGPT shouldn't generate you a phishing email or some malicious code that you can use to hack an application. Right. It has some standards, but in the past, for example, some of those were even be able to. You could bypass that if you provide it in base 64 encoded and then it returns you really that what you want to have. And so you can always bypass that. Sometimes especially as soon as there is a new model, it might again start from the beginning. That's the first thing. And also the second thing is about. So I talk about jailbreak. Right. And the second one is prompt injection. So you know the also like GPT scans websites and so on. Yeah. I then might have like in a website, maybe an image that looks for me like an image but for GPT, it's a command to forward you to maybe another page or do something else that you can't see as a human. Right. And so that's also I think where we need to be really careful before we, when we use an AI to be aware of that. And I think that especially also quas can bring values into that to understand it kind of to try even to break in this case the AI in a way or try to highlight some security Issues based on that and maybe on very simple example, let's say we have a web shop and implement a chatbot. I as a quave would first take a look at that chatbot shouldn't generate me some python code, let's call it like that. Right. That wouldn't be very good. Or that chatbot shouldn't leak some data that maybe are hidden for other users, which is also very important. And I think that's something also as a qua with our like mindset on how we approach problems, it's definitely something we also can test and cover.

That's a great point. I speak to a lot of companies that are starting to build ChatGPT into their applications and I just thought of it with you thinking like it could be trained on data that you don't know that someone leaked a secret or something or even a credit card number for testing that that's valid. And so if you don't test that in that way as a hacker and security that you're going to be vulnerable. So it seems like it's going to be even more demand now if that's something your company's dealing with.

Yeah, definitely.

Awesome. So Boris, you got yourself trained up in security, getting more involved. How can you have QA maybe enable themselves to be more involved in security? Do you have like an approach or what you did that you think would work for them as well?

Yeah, so also that you previously also mentioned OWASP top 10 and I think that's for example a good start, especially when you work in web based applications. So this is because there are so many different things and you really first need to find the focus, where is your place and typically it's ovasop then and there are a lot of trainings that you can do on udemy that there are plenty of trainings. What I can also recommend aside from that is for example a typical UDEMY Training on the wastop 10 is also for example to try Hack Me academy. Quite often within this academy you have like it explains you the concept, how it works and then you immediately can test it out on a real environment. So it's of course a mocked environment, so it's behind VPN everything but you can train it out and really immediately apply to understand how that works. So and in addition to that, what I also would recommend is when you want to practice it, there is also the OWASP top 10 choice shop that you can host locally on your environment and then you can really try it out, play around with it and really try to Maybe see how let's say this SQL injections work for example or some other threads that are basically happening there. And as a final there are like two more points that I want to mention. There is also the crappy API vulnerabilities in kind of local environment that you can run where you can really then also test it on your own from APIs. Try to hack APIs in a way. And there is also a pretty good book I would recommend it's Hacking APIs by Corey Ball. It's really nice book that also tells you with examples where you can really use the creepy API to really try it out, how it works and so on. And it's really nice and well structured and really help also quest understand about API security. And for me API security is still one of the most important crucial things in the web development as like with microservices and everything. API first and yeah it's good but we need to make them also secure and in a way. And as final point I would also highlight hack to Box. So if you really kind of gained some basics and really want to try it out on a let's say environment, then I would really recommend hack to box. It has a lot of gamification elements where you can progress for let's say a script kitty to a hacker, then to an elite hacker or something like that by kind of hacking different environments and so on. It's a lot of fun. It's maybe time intensive, that's for sure, but that's also a lot of fun to understand, especially how maybe this hacker or this hacker group actually might try to penetrate your application in how what's their approach and so on. And this is really also super helpful to understand the methodologies of hackers.

Awesome stuff. Okay Boris, before we go, is there one piece of actual advice you can give to someone to help them with their security testing efforts and what's the best way to find or contact you.

The actual security advice is really to take cybersecurity seriously and to think on this on a holistic level. So it's not also just quas. It's not developers responsibility, not just. It's also not just cybersecurity experts responsibility. It should be the whole team responsibility. And that's also why we as QAs also have to keep up with that with some security fundamentals to be able to achieve that and how they can contact me. I would say feel free to reach out to me for example on LinkedIn or any other like on my email address I can also provide it and also happy to help in any, I would say possible way I can.

Thanks again for your automation awesomeness. The links to everything of value we covered in this episode. Head on over to testguild.com a514 and if the show has helped you in any way, why not rate it and review it in itunes? Reviews really help in the rankings of the show and I read each and every one of them. So that's it for this episode of the Test Guild Automation Podcast. I'm Joe and my mission is to help you succeed with creating end to end full stack automation awesomeness. As always, test everything and keep the good. Cheers. Hey, thank you for tuning in. It's incredible to connect with close to 400,000 followers across every all our platforms and over 40,000 email subscribers who are at the forefront of automation testing and DevOps. If you haven't yet, join our vibrant community@testguild.com where you become part of our elite circle driving innovation and software testing and automation. And if you're a tool provider or have a service looking to empower our guild with solutions that elevate skills and tackle real world challenges, we're excited to collaborate. Visit testguild.info to explore how we can create transformative experiences together. Let's push the boundaries of what we can achieve.

With lutes and lyres, the bards began their song A tune of knowledge, a melody of code through the air it spread like wildfire through the land Guiding tester showing the secrets to behold.