Hacker Tools and Skills For Security Testers with Jahmel Harris

Five, four, three, two, one. Drop that.

Welcome to the Test Guild Security Podcast. We're all getting together to learn more about security testing with your host, Joe Calantonio.

Hey, it's Joe. And welcome to another episode of the Test Guild Security Podcast. Today we're going to be talking with Jamal Harris, all about hacker tools for developers and testers, how to add security tests into your pipelines. So this interview is based on a session Jamal did for us at the 2019 Secure Guild Online conference, as well as the live Q and A that followed that session. Jamal is a security researcher and hacker. He co founded Digital Interruption, a security consultancy which helps secure organizations with a mix of penetration testing and helping to embed security into application development pipelines. Jamal has a background not only in security testing, but also software development, and he's advised engineers on balancing security with function. Jamal also has a particular interest in mobile application security reverse engineering and has spoken at a bunch of different workshops in the uk. He also runs the Manchester Grey Hats, a group aiming to bring hackers together to share knowledge and skills. If you want to learn about hacker tools and how to get started with security testing, you don't want to miss this episode. Check it out. It's no secret that the number and complexity of applications are growing. But what is alarming is that 90% of security incidents result from exploits against defects in software. The need for better application security has never been higher and Micro Focus Fortify is here to help. Fordify is the recognized market leader in application security and it is the most comprehensive and scalable application security solution that works with your current development tools and processes. With Fortify, you could start security applications in a single day, including custom code, open source or commercial components and scale as your needs grow with an on premise as a service or hybrid implementation. Check them out. Head on over to micro focus.com app security. Hey Jamal, welcome to the Guild.

Thank you.

All right, Jamal, before we get into it, could you just tell us a little bit more about yourself that I may have missed in your intro?

I'm a penetration tester at Digital Interruption and over the last few years I've been attending more QA testing events. These have been fantastic and the one thing I quickly noticed was the amount of work being put into automation. Unfortunately, this hasn't quite happened yet in security. We do have some great tools, some fantastic bits of software that we use to help us with security testing, but they are very much tools for hackers by hackers, if that makes sense. They often have clunky UIs or sometimes archaic command line interfaces. So I believe that we need more hacker tools for developers and QA testers.

So, Jay, coming from a functional testing background, we have a lot of tools to help us with automation, but I don't see as much with security testing. Do you see any reason why that is? Any issues with pen testing that makes it so? It's hard.

One of the disadvantages of a pen test is that a pen test is very much a point in time. We have the ability to say that this pen tester found these vulnerabilities when they looked at this application at this point in time. And even if these vulnerabilities are fixed, there's no way to know whether they're going to be reintroduced at a later date. Unless, of course, we redo the pen test. The alternative is to try and build our own tools that will try and perform these checks when the application is built and plug these into some kind of build pipeline. As I said, we do have some great tools for security testing, but either they're used to do exploratory testing. You might have used like Burp Suite or Zap or something like this before, but the require a certain amount of specialist knowledge to perform security tests. Or on the other end, they're quite simple vulnerability scanners that only detect very generic issues.

Very few tools exist that actually let us build our own security test cases.

So, Jamal, before the show, you told me about a tool that you use all the time for this. I believe it was called Frida. So I guess at a high level, what is Frida?

Firstly, Frida is a framework or toolkit for instrumentation, also known as application hooking. On the Frida website it says inject your own scripts into black box processes, hook any function, Spy on crypto APIs, or trace private application code. No source code needed.

I guess my next question is, what's application hooking?

So application hooking means we have the ability to change how an application works at runtime by injecting our own code into the process. Effectively, this means we can have our own code run instead of the original code, or we can call functions internal to an application whenever we want.

Jamal, can you give like a real world example how you would use this.

For a practical example of how this might be useful in a pen test? I once tested an application that was encrypting messages before it sent them over the network. Now, obviously this made it difficult to look for vulnerabilities in the backend, as I didn't know the message format. But using Frida, it was possible to hook the functions that encrypted the messages and not only see the unencrypted messages, but to change these before they were encrypted. So Frida would allow us to hook an application and to call internal functions as if we were part of the program. For example, if the application had an encrypt function which encrypted data entered by a keyboard, we could call that function anytime we liked with our own parameters. This would be helpful for automating some tests or calling functions with invalid data to see what happens. Frida also lets us change runtime behavior by calling R code rather than the original application code. For example, if we want to see the internal state of a program, we could hook the encrypt function and change it so that it logs all parameters, then call the original encrypt function if we needed to.

So, at a high level, how does Frida work?

So just quickly. Frida works like this. It injects the JavaScript v8 engine into the process we want to hook. This could be a remote process, such as on an Android device. This requires the Android device to run the Frida server. On our host, we run the Frida client, which pushes the JavaScript code over to the server. It's in the JavaScript that we define what functions we want to hook and what the new function logic should be. The Frida server and Frida client communicate using JSON, so it's possible to send messages from the server to the client by using the send function in JavaScript.

So how is this helpful for actual security testing? What are some scenarios you would use.

This for outside of pen testing? This type of technique can be useful for forcing errors into an application, such as injecting sleep or reading specific data from a file or a network. Or, as we saw here, we can inject login into an application so we can see the internal state. So in this example, we're going to take a purposely vulnerable Android application that displays a number of vulnerabilities that I've seen in real pen tests. This application, it protects user data, such as passwords and notes, by encrypting them. And to use the application, the user needs to input a pin. Now, in this case, the vulnerability is that there is no brute force protection, so an attacker would be able to keep trying PIN codes until they were able to access the data. To exploit this vulnerability, an attacker might plug in some hardware into the Android device that would mimic a keyboard and therefore keep trying different pins. So this kind of attack would actually be feasible. So let's assume we know this vulnerability exists either because we discovered it during testing, or even better, through some kind of threat modeling exercise, or maybe even by reviewing the results of a pen test report. So we want to use Frida to write a test case that will try to brute force the application and report whether the test was successful or not. That is whether the vulnerability exists or not.

So to actually see Jamal's session at Secure Guild, all you need to do is go to guildconferences.com, click on our events and click on Secure Guild 2019 and register for an after event ticket. Like I said, this interview was cobbled together from talking to Jamal directly, and also some sessions he did for us for Secure Guild and also a live Q and A we had with Jamal as well. So, Jamal, are there any other features or scenarios that Frida can help us with?

So obviously we looked at security testing in this case, but really Frida could be used for any type of testing. One of these advantages of Frida is that we're not really looking just at simulating UI things. So we have the ability to test, for example, our performance issues. Using Frida, we might be able to add sleeps into our code at different points. We have the ability to add logging into our applications or to maybe even change the data that is being read from a file, for example. Really, it allows us to change how an application works at runtime. Ultimately, we can perform any testing that makes sense for our applications.

I am new to security testing, but this almost sounds like mocking that you would use for say, a functional test. Am I understanding it correctly?

Yeah, I mean, that is one way to think about it. So we have the application. Once we understand how the application is put together, that is like the functions and the methods and objects are in place, we have the ability to swap those out for our own versions of them. One thing that I like to do with it is if I have an application that is production ready, that doesn't have any logging in or something like that, we can literally just ship out these free the JavaScript files and then add those into the program.

So we have a way to test our production version of the code without having to have separate debug builds or config files that could be on and off. You're totally right. We have the way of just instead of where you would traditionally mock a database or something to provide test data so it doesn't actually connect to a real database, we have the ability to do that, but kind of for every resource and Even the internal things we might decide that we want to. In the example before, I discussed encryption functions. So we have the ability to change kind of even how that encryption function works or the data that's going into it or being returned from it. So, yeah, I mean, that's kind of why it's such a powerful tool. And I feel like it's kind of a shame that it's only really used by pen testers and security researchers when I think if more people were to use it, they could actually see how they could use it to automate so much more of the test cases that have traditionally been fairly difficult to automate.

So, Jamal, how is Frida different than, say, a functional application like Appium that's really popular for testing mobile applications? For functional testing, sure.

So, as I mentioned, the big difference here is that Frida is not just simulating the ui. And also we're using Frida here as one specific example of hacker tools for hackers. I like to describe them. This kind of process of trying to add an API or to control the tool set that you need to use is kind of a powerful thing. So there are so many tools out there that are used by pen testers and actually they need to be integrated into.

QA testing. But, yeah, this technique differs in that we're not just manipulating the UI or recording screen presses or inputs. We have the ability, as I mentioned before, to change how the application actually works without having to change the code of the application. So I think that's really how it differs. The other thing I would say actually is, unfortunately.

It'S not as mature as some of the tooling that testers might be a bit more familiar with. It does require some amount of massaging to get it to work correctly. Again, I think that's one of the things that security and testers can do to work together to develop tools that can capture our knowledge, but can be used in a way that is relevant for people that aren't just pen testers.

Is Frida only for mobile testing?

Frida can work on lots of different platforms. I use it a lot for Android and iOS applications.

It really gives me the power to, for example, disable security controls when I'm testing a mobile application, which for me becomes really useful. Or there might be a way I need to simulate some malicious input when I might not really be able to do that because of testing constraints. So I use it a lot on mobile, but it does actually work on Windows and Linux as well, and Mac and actually QNX. So quite an obscure operating system there. So the BlackBerry OS.

And QNX is used for a lot of other embedded systems as well. So I think between all those it tends to be something that works on most things. It doesn't work with. NET applications, or at least unless that's changed recently, I've not been able to have it work on. Net, but that's okay because I think there are other tools we can use for. NET for doing similar things.

So one complaint developers have is when you add more tests, you slow down the CI CD pipeline, which makes the feedback loop even slower to give your developers feedback. So would you add these free to type tests to your CI CD pipelines and if so, how does that work?

Yeah, so good question. It really depends a lot on.

At least with security, on risk appetite and on the actual test itself. So some of them I wouldn't suggest running all the time. This brute force one, because it's at least the way we wrote the test is actually trying to brute force the application. So there will be some amount of slowdown in that, but it's a slowdown that you can measure, which is quite helpful.

As I said, because we are actually trying to brute force the pin. We can calculate.

How many times the PIN would need to run and we can calculate that and decide on whether we want to run that all the time or maybe just weekly or nightly. My recommendation for most of my clients is to run security tests weekly. I don't think it needs to necessarily be done more than that, especially when they're not the scanning type of checks. With this case we have a test that we write a test case for. There's not necessarily a reason to run that super often. So I tend to recommend weekly or I would say whenever there's a large change to the application. That's the other time I tend to recommend security tests. So yeah, really good question and I kind of. I wish there was one answer. It depends a lot on how long the test takes to run and risk appetite of the company and I guess things like how many developers are working on the code as well, if it's something that lots of people are changing, it probably makes sense to run a bit more often.

So looking at Frieda, it does take a little bit to get up and running. So is there anything that makes it easier to get up and running?

So. Not at the moment. It is. I mean, I agree it is a bit complex and even to be honest, I know other pen testers that avoid doing this kind of thing. It's not as easy as it should be. So I actually mentioned in that talk that we're developing some tools that we think can actually help with this.

We're actually looking for people to help us with at the moment. So if anyone watching this is interested and wants to help us develop this in a way that makes sense for people that might be using it, ie, testers and developers definitely get in contact.

So, yeah, there's nothing that is really easy yet. But that's why, personally, I'm trying to involve myself more with the QA testing community because it should be easier. It should be a lot easier. But for some reason, I think pen testers, we like to make things seem more complex than they are. So we like to have tools that have command line interfaces and have nice ASCII banners on the screen and things like that. I think we forget really the learning curve for some of these things. And so there have been times I've been on a pen test, for example. Sorry, I've been preparing for a pen test and I've just said to the, to our customers, oh, if you just, you know, run this tool and do this thing and get me this information. And they just look at me like, what is this strange, archaic tool? And then I remember, actually, yeah, it took me like a year to learn how to use this thing. I can't exactly expect someone to just run it. So, yeah, it's not as easy as it should be. I definitely agree. Hopefully over the next year we can start to get there. If the community start working together and building tools. As I said, not just hacker tools for hackers, but hacker tools for developers and testers, I think is significantly more important.

So besides Frieda, are there any other tools you use daily for your security testing efforts?

So for Android and iOS, I wouldn't say there's really many easy tools. Burp Suite and Zap are ones always in my toolkit, but again, they're more for exploratory testing. I know Zap does have an API that you can use to start to kind of integrate it into Jenkins, like we saw with Frida. But, you know, you can do that for Zap for web API testing as well. I think there's Drozer for Android, which again, is a bit. A bit of a challenging tool to use if you don't do a lot of security testing for Android. The rest of the stuff, to be honest, is stuff that doesn't really help so much with automation, but more about manual testing, so exploratory testing. Sometimes I'll even go as far as having to reverse Engineer a binary as well. So they might say here is a Android application and I'll have to actually take it apart and understand kind of where the sensitive things are or try to recover passwords maybe that are stored inside the application. Actually I say that's one of the benefits of taking security into the QA testing role because QA testers when they're in a team will understand more about the product than I necessarily will in the five or 10 days I have for the test. So then you can actually get rid of a lot of the testing tools that I use because that knowledge will already be within the team. So yeah, there are definitely tools that are worth playing with. You mentioned Fiddler as well. So it's not one I use but I'm aware of it. It's a proxy similar to Burp Suite and to Zap. If that works for you, that's cool. Things like.

It can be hard as well understanding which technologies are in use and trying to find tools that are specific for those things. But again I think the real benefit is not necessarily just trying to learn how to use the tools I use, but let's try and develop new ways of using some of these tools or new tools that fit in different workflows.

Jamal, as I mentioned numerous times so far, I'm new to security testing so something just came to my mind was you talked about Tools for Android iOS so when you're creating a security plan I didn't realize you had to test all these other things for security. There's not like a security exploit could be anywhere I guess. So how do you know where to focus your efforts first? That gives you the most bang for your buck.

So I tend to focus on things around authentication. I tend to look for the low hanging fruit first which to be honest is one of the another issue with pen testing that I see. Unfortunately I do need to look for the low hanging fruit so because to not report it would would be almost negligent. But that's also not the best way to spend security budget on a pen tester looking for low hanging fruit. So that is the stuff that I tend to recommend. The things that are easy to find, things that are documented, things that are literally security checks. For example you could for Android you could, you should make sure that you know what the application is exporting. You know Android has this concept of IPC endpoints kind of openings in the application to allow other applications to communicate with them. If you know what your application should expose, make sure that that's always the case because when you give it to a pen tester, we have to look through them all and decide whether there's any security risk there. So the larger that list is, the more time to spend looking on those things. So there will be things that are kind of documented that you should check for, things around encryption. That's another thing that I look at. Oftentimes I deal with my clients and they give me an application and it's storing data in the sandbox, but it's encrypted. The question is, where are they storing the key to decrypt it? Is that if that key is hard coded, which I've seen plenty of times, then the encryption doesn't really add much to the application. You know, it's sure it's stored on disk and encrypted form, but if the key is there, I can then decrypt it. So it's kind of about those things. And the other part of it is this thing that takes a bit more, I guess, experience to try and detect, and there's no real way to.

Find those. There's this concept that I heard recently from someone called Claire Reckless that she came up with called security smells. So similar to code smells, but more about security. And I really like the idea because, you know, over time you start to get a feel for where some of these security issues are. You know, like if the application is doing this, then, you know, it's likely doing this in the background. Or if you know, if this thing is true, then this other assumption must be true as well. So, yeah, that's more about just trying to get a understanding for the types of vulnerabilities that tend to be common with certain applications and.

How to notice them when you do come across them. Because it can be easy not to. It can be easy to not realize that something is vulnerability.

So another thing that comes to mind is with mobile testing, a lot of these Android devices have different OS flavors. Does this come into play when you're thinking about security like it does when you're thinking about functional testing?

For some tests, yes. For example, I mentioned again, encryption. I like that example because people think encrypted means safe and secure isn't always so. If I'm looking at something, whether it's iOS or Android, there's a potential that the key could be hard coded in the application, which would be a risk. But there are definitely things that are relevant to one platform or another. The implementation stuff, for example, is a bit more relevant to one there might be issues with specific to Java, for example, or how iOS handles packages, or how Android lets you define IPC endpoints. So there are definitely differences. I would say that there is more overlap though, in terms of the types of vulnerabilities that you're looking for, how you might test for those, or what the. The kind of intricacies are some slight differences, but the concepts at least overlap quite heavily. If you know how to do it on one, you can quite easily figure out how to do it on the other as well. The other thing that might be different is how you actually go about manipulating the device in a way, I guess. So with Android you can use ADB and you don't necessarily need to have a rooted device do a lot of testing. With iOS, you tend to require a jailbroken device and you SSH into it and you can manipulate it that way. So again, like a similar thing, you know, both ways to communicate with the device and to run your own commands and to start interrogating applications, but the actual way you do it is slightly different for each jamal.

I guess another thing, the kind of soft skills needed for security testing you mentioned there's a lot more awareness for automating automated test and you know, reluctantly. A lot of companies do it just because it's part of the testing process, but with security, it still seems to be like an afterthought, oh, we got hacked, now we need to test. It never seems like we need to do this or you know, it's going to be bad. I mean, as, as someone that offers these services, like how do you get awareness to know that or buy in from developers and management that security testing is needed and is obviously critical before something bad happens.

It is one of the hardest things. Honestly, it's so difficult. It's something I really believe in. I think that pen testing is definitely a fantastic tool, but it's just one tool. It's only one way to go about doing security. And if it worked as well as we claim it does, then we wouldn't see the types of breaches that we are seeing. So to convince people that they should be considering security early is definitely one of the big struggles I'm having.

Really. I just, I try and keep telling that message to people and slowly over time, people are starting to listen. I remember a couple of years ago when I first started talking to QA testers and I said, you know, who here in the room is working on security stuff within their tests? And someone said to me that it's not part of their job, that's someone else's job. I'm like, oh, okay. And then actually, you know what? I think six months ago, they messaged me on Twitter and said, you know what? I was totally wrong. Security is definitely part of what I do. And they're actually one of the people that are, where they work, they are pushing security testing. They're trying to come up with ways that the QR testers can do security testing, they can automate stuff. So I think people are kind of, at least they are aware of it. But there's this fear a lot of the time that you need to be an expert to do security testing. And I don't think that's true. I mean, we saw that the vulnerability that we looked at earlier was just a brute force test. You know, can we try to log into application and not log us out?

So that you don't necessarily need to be an expert to do security testing. And once you start to embrace that, you can start to look at other ways to develop applications. So DevOps is obviously a big movement. There's this idea of adding security into it. So dev set ops, adding tools like Frida potentially into the pipeline, adding other things in there that can help to automate some of these checks. And then we get to the point where pen testing is not the only way to be secure. And of course, pen testing doesn't make you secure. You need to fix those issues. Any issues that it detects, you need to actually fix those to be more secure. And so anything that we're doing to make that easier or to make security another measure of quality, the more likely we are to develop applications that are secure. And as is, I'm a user of applications, I want to make the process easier because I want to know that my data is more secure. If I'm signing in or logged into a service, I don't want that data going out to who knows who. So, yeah, I think we just need to make the process easier. The QA testing community needs to start to understand that they can do some of these things. And I think in the security community, we need to stop being these gatekeepers that tell people that you can't do or you need to be a hacker to do these things.

All right, Jamal, before we go, is there one piece of actionable advice you can give someone to help them with their security testing efforts?

All I would say is, yeah, a big part of what we're doing is trying to close this gap. So I really encourage testers to go to security conferences, you know, show us why what we do is important to you. You know, tell us what we're doing right, what we're not doing right, and then for anyone listening, that is in security, make the effort to go to other events. You know, go to developer conferences and testing conferences. I've learned so much just by going to testing conferences. Honestly, the way that testers work is so different to how I do things, and it's made me a better security tester.

Thank you Jamal for your security testing awesomeness. Links to everything of value we covered in this episode. Head on over to testguild.com S08 and while you're there, make sure to click on the Try It Today link under the Exclusive Sponsor section to learn all about Micro Focus Fortify, which is the recognized market leader in application security. Really cool tool. You definitely want to check it out. So that's it for this episode of the Test Guild Security Testing Podcast. I'm Joe and my mission is to help you succeed with creating end to end full stack automation awesomeness. As always, test everything and keep the good. Cheers.

Thanks for listening to the Test Guild Security Podcast. Head on over to testguild.com for full show notes, amazing blog article articles and online testing conferences. Make sure to subscribe to join the Guild and continue your testing journey. This has been a Joe Calantonio production.