How to Security Test Your APIs with Troy Hunt

Five, four, three, two, one.

Drop that.

Welcome to the Test Guild Security Podcast. We're all getting together to learn more about security testing with your host, Joe Calantonio.

Hey, it's Joe. And welcome to another episode of the Test Guild Security Testing Podcast. Happy New Year and let this be the year that we really get security testing as part of our software development life cycle. Today we're going to be going back in time to hear an interview I did with Troy Hunt on how to security test your APIs. This is actually one of the most popular episodes I ever, ever put out and it was really early in my podcast career. I don't even think I'd be able to get Trey Hunt to respond to me nowadays. So that's how long ago this was. But the content is still relevant and we're going over APIs, which I think is going to be even more and more popular in 2020, especially as more IoT devices. For those that don't know, Troy Hunt is a Microsoft Regional Director and MVP for Developer Security and a full time author for pluralsight. That's where I actually first heard of Troy and this episode was based on his course Hack youk API. Since I interviewed Troy way back in 2014, he's become an industry leader in the security space and produced more than 20 top rated courses for Pluralsight. I even think he appeared before Congress to talk about security, so really knows his stuff. And I'm still really excited about this content. That's why I want to share share it with you since I think it's really relevant, especially as we go into 2020. Check it out. It's no secret that the number and complexity of applications are growing, but what is alarming is that 90% of security incidents result from exploits against defects in software. The need for better application security has never been higher and Micro Focus Fortify is here to help. Fortify is the recognized market leader in application security and it is the most comprehensive and scalable application security solution that works with your current development tools and processes. With Fortify, you can start security applications in a single day, including custom code, open source or commercial components and scale as your needs grow with an on premise as a service or hybrid implementation. Check them out. Head on over to micro focus.com Apple Security.

Hey Trey, welcome to the Guild.

G', Day Joe. Thanks for having me.

So before we get started, could you just tell us a little bit more about yourself?

Yes, I am, as you can probably hear, an Australian. I am an Australian Microsoft Most Valuable Professional for Developer Security, which is a Very nice award they give me. I'm not a Microsoft employee, but they give me a nice little recognition for things I do in the, in the developer security space and particularly in the Internet security space of web apps and APIs and all things. HTTP awesome.

Yeah, I'm really excited to have you on the show today. I recently took one of your pluralsight courses, how to hack your APIs. Can you tell us why you think API security is so important?

Yeah, I guess there's a few different things there, Joe. So one of them is that when we look at the web holistically, vulnerabilities in web applications in many ways are much easier to find than in APIs. So a really good example is the use of SSL. So you go to your bank, you see HTTPs in your browser, you see a padlock icon, you can look at the certificate, you get this sort of assurance that your credentials and then all the information coming back is going to be secure. Now flip that over to the mobile app world where you've got a custom built app that's doing some sort of web communication in the background and you get none of that. So you get no assurance of who it's talking to. You get no assurance of if it's talking to it securely. So that's one thing. The other thing is that these apps tend to gobble up a lot of nasty security implementations. So for example, on ssl, again, if you go to a website in your browser and the certificate is invalid, there's bells and whistles and bright lights that go off everywhere in your mobile apps. You see nothing. And as it turns out, a lot of mobile apps are either not using SSL correctly or at all, or they're using things like invalid certificates. So they basically got very, very poor implementations. So that's one. And I guess the other thing that's really significant here is that we do have a bit of a gold rush, if you like, of Internet connected things, whether they be apps on mobile or whether they be the physical things in this new Internet of things world that we're entering. And what tends to happen when we have this gold rush of people trying to rush stuff to market is shortcuts get taken around things like security. So I guess a combination of that sort of little bit more abstraction between the consumer and the actual services that are being hit and the rate at which these things are going out makes it a little bit of a perfect storm for vulnerabilities in our APIs.

Great. Yeah, I definitely agree it's a timely course. I read somewhere that in 2020 there'd probably be over 50 billion objects connected to the Internet. And I assume most of these things would be headless API type transactions going on. So what would you say are the high level concepts a test engineer new to API security testing should know in order to get started?

Look, a couple of things and I think that one of the most significant things that will be new probably to a lot of testers, certainly testers that are more, I guess, functionality centric, and certainly something that's new to a lot of developers, is that all of the communication that your device is having with these HTTP based APIs is, can be observed, can be intercepted and can be manipulated. And that to me is sort of what I see being the light bulb moment for most people where they go, oh, okay, so every time this app, which I've built and I sort of quote unquote own, I trust is talking to these services that are also built and I own, yet someone is able to sit in there in the middle and see what's going on. And not only can they do that for HTTP based communications, but they can do that for HTTPs based communications if you install a self signed cert on the device. So I think the number one thing here for testers and developers alike is be conscious that everything is for all intents and purposes in the public domain. So that's where I'd start. And I think then in terms of testers, it comes down to understanding what are some of the common patterns of risk or really just poor implementations in many ways that we find in these APIs. So things like insufficient authorization. And a really good example here is, is the app trusting that the user is successfully authenticated before it shows things, but those things have already been sent down over the wire. So a really good example that I often use of this because it's, it's not sort of too sensitive, is British Airways. The British Airways mobile app, when you open that guy up, it goes out to the Internet and it pulls down all the WI fi passwords for the lounges. Now you're not meant to see those until you actually authenticate to the app yet it's already grabbed them, pulled them down over the connection, stored them in the device, expecting that you'll authenticate, but you can just jump in the middle and intercept them. So they're the sort of patterns that I, that I often see that I think testers need to be aware of.

Your course is really eye opening. I would recommend to anyone listening to head on over to plural site and Sign up for a month and just try it out. This course is awesome and so eye opening because you actually give real world examples and you walk us through and show us how easy it is actually, like you said, to intercept the messages and communication going on. And you know, I was just shocked on some of the things that you can actually find out about. What is a man in the middle attack? It seems to be one of the more common security attacks a hacker would use.

It's certainly one of the things that we most commonly defend against. So a man in the middle in a very broad context is effectively some malicious party sitting between a communication that's happening between two parties. So if we think about this in a, in a web context, if you've got a browser talking to a server and it's connecting out via say your home router and then out to your ISP and then across the Internet to a server, what happens if the ISP is monitoring your communications? And we see this happen a lot overseas and ironically we know it happens in the US now via the nsa and I'm sure that if our government has enough money, they're probably doing it too. So a man in the middle is where there is someone sitting in the middle of that communication. And of course this is why we have this thing that is SSL or TLS or HTTPs. We tend to use those terms a bit interchangeably where we have end to end encryption of the connection. It's to ensure that if anyone does sit themselves in the middle of that communication, they can't either observe what's going on and they also can't manipulate what's going on. So a lot of people forget it's not just about making sure that your things aren't seen, it's that your things aren't changed as well.

I think another example you gave in the course is how you can intercept messages from your mobile phone, but also things that you wouldn't think of. You have an example of an Internet scale, how even though you can't interface with it directly, there are ways that you can still hack into that and find the communication that's going on between it.

Yeah, so this is where it starts to get really interesting because as you mentioned before, we're getting all of these Internet connected things. We're entering the world of IPv6 and everything with publicly addressable unique IDs and we're sort of getting this point where I'm not sure why, but our fridge needs to talk to, our toaster needs to talk to the kettle and all of these things are sort of connected. And of course the interesting thing here is that all of these connected things, when they do talk back to services centrally, that they do tend to talk over HTTP or HTTPs and all of those things are interceptible. So even though we might not be able to say, go to my bathroom scales, I've got a set of those, those nice wything scales. I can't go to proxy to send the traffic through Fiddler, like I show with mobile devices in the course, you can still intercept that traffic. So I could arp poison the device. So have that device route its traffic through another intermediatory which then dumps all the packets that are going through. I could connect that device to a malicious wireless access point. So you mentioned Pineapple. There's a device called a wifi Pineapple that acts as an access point that you can connect to like any WI FI connection and dump every packet that goes through it. And we can then inspect that so the message there for something like the scales or you can even get a connected fork. These days all of these things are connected. So for all of these things we can look at what are the services that talking to, what's the data it's sending, and also how the service is responding. All of that can be observed and all of that can be probed.

Interesting. So someone listening that might be new to this or haven't seen the course might say, big deal. Of course all the data is encrypted, so who cares? If people could see my, the communication going on, what would you say to that?

So we've got to separate the issues. So one of them is confidentiality. Insofar as can a man in the middle, let's say the nsa, because it's still kind of topical a year later, let's say the NSA is looking at what my body weight is. So we're not too worried about that. At least I'm not too worried about that. I don't really care if the government wants to see my weight. What's more interesting though is if I'm an attacker and I have the scales, a set of scales, and I actually look at the traffic patterns and I look at the services that it's calling and the data that it's sending, could I possibly then exploit those services? So once I've discovered them, for example, the service which brings back data, can I change my user ID to someone else's user ID and get their data? Okay, so can I actually subvert the authorization protocols that are in there and pull back Information I'm not meant to get access to. So that's the sort of thing that we're more worried about. In a case like the scales, what.

Tool do you normally use to capture this information?

So the easiest tool that anyone can use to go and test this sort of thing now is Fiddler. And Fiddler is a pretty popular tool among many developers, and many of them use it for debugging so that they can look at the HTTP request that the PC they're running it on is sending to a service or to a website. And what a lot of people don't know is that Fiddler can actually act as an HTTP proxy. So when you go into the settings in Fiddler, there's a Connections tab and you can say, allow remote connections. Now, as soon as you tick that guy, you can go to your iPad or any other device that you can configure a proxy on and tell it to use the IP address of your PC as a proxy. So effectively what's happening now is everything you do on that device is now routed through Fiddler. So just like you can open Fiddler and see all the requests from your PC, you can see all the requests from this external device, and that's how simple it is to actually find how the device is communicating with the backend. Now, the other thing that that then leads onto is that you can use other tools within Fiddler to probe away the way the service is operating. So, for example, you can use Fiddler script to modify the request or modify the response. So in the course, I show how when we log on and we get a response that indicates I'm not an administrator, what happens if we use Fiddler script to change that response and say, look, I am an administrator. How does the app respond differently and is there a vulnerability in that?

A lot of this seems kind of manual. Is there a way to automate security testing, or is there a way to run known attacks, then maybe a set of a test suite of security tests you can run against your APIs, or is it really more of a hands on kind of hacking away to discover what those particular holes might be in the API that you're developing?

It's a bit of both. So certainly there are tools that can automate security analysis. I wrote something very recently about a tool called netsparker, which is very good for testing website security, where you just plug in the root URL of the website and it crawls through and makes tens of thousands of requests and looks for common vulnerabilities. You can use that to some degree for your APIs. But the APIs don't tend to be discoverable in the same way that resources on a web page are. You don't have hyperlinks to resources that can then be easily discovered. So I think for testing APIs, it is a more manual process. Some of the automated tools are still good for looking at say server configuration vulnerabilities. So is it leaking information? It shouldn't in terms of say, internal error messages or excessive headers or things that it doesn't need to disclose. So the tools are good for that. But particularly for APIs, you really want to look at what is the device doing, how's the device communicating, and then try and probe away at I guess, really the business logic and the semantics of how it works. And that's something that's a lot harder to automate.

This brings up another question I think about, especially for a team that's new to security testing. They have a two week Sprint and they have a lot of things to have on the definition of done, one of them say being security testing. So where in the software development life cycle do you think security testing should be performed? Is it a one time operation or is it an ongoing event that you normally do throughout the lifecycle?

Yeah, everywhere. That's the easy answer. So look, it often tends to be a very retrospective sort of event. But one of the things that we know from building software is that the further down the life cycle of development, a change is made, the more it costs. So for example, if we can change the way we want the thing to operate in the requirements phase, it's much cheaper than in the design phase, which is much cheaper than in the development phase, which is a lot cheaper than the test phase. So the earlier we can make these decisions and solve these problems, the better. So for me that boils down to a couple of different things. And one of them is having well trained developers that actually understand the security paradigms. And that's really why I write these courses, because those are the guys that we most want to influence. Because if we can get those guys right when we do security scans, right at the end of the project, we're going to find a whole lot less stuff and it's going to be a whole lot cheaper to fix. Now, we should still do that. And I do like the idea of having dedicated security professionals who may not be developers and frankly many times may not even be on the same planet as them. But they come at things from a very different angle with a different view, and they usually find things that the developer won't. So I think it's good that they are in the process as well. Now then becomes a question of are we working in some sort of iterative agile fashion where we can do that multiple times throughout the project? That's always a good thing. Or at the very least, let's make sure we do that at the end. And of course the lifecycle of an app doesn't just end when we release it either. So is there any sort of continuous monitoring? The number of times I've seen someone push a change where they just change the error configuration to debug something and then six months later it's still spitting out internal implementation data. So an ongoing continuous analysis is also a good idea.

So Trey, besides Fiddler, what are your thoughts on Wireshark or metasploit?

So they're both very interesting tools. So Wireshark for packet capture and analysis. So you're really getting down beneath just the HTTP level, which is where Fiddler tends to live, and you are getting down to individual packets across all sorts of different protocols. So that will get you right down to the metal and it will get you a lot of detailed information. It's harder to get head around as well. It's harder to digest than just one HTTP request because now it's a series of packets that you now need to sort of associate back together and draw some conclusions from. So very good stuff, but requires some attention to get the most out of it. And metasploit being very penetration test centric, it's a pretty serious product made by a company called Rapid7. I've seen some of what Metasploit does and it scared me. It confused me as well at times. In many cases we are getting down to some very, very low level stuff. But what tends to happen with metasploit is that when a vulnerability arises, they get modules built onto it pretty quickly. So we're talking about this the day after a fairly serious vulnerability has been released to the web in terms of this Bash bug. Inevitably we will see a Metasploit module. If it's not there already, I'd be quite surprised. Which will help penetration testers find at risk systems. So it's a very, very full on piece of kit aimed very squarely at security professionals.

Great. But it almost sounds like Fiddler. To get started is more than enough. It'll get your team talking about security and finding issues probably right off the bat. And I believe in your first hack yourself first you actually go into it in detail how to Use it. So it gives you a good overview to get started.

Yeah, correct. And the reason why I advocate Fiddler is that for the audience I speak to, which is developers, it's a familiar tool that you can get your head around very quickly. I think if I sort of introduce developers directly to Wireshark, they would kind of go, oh my God, what is this?

Which is what I often do when I open it up. So I think Fiddler is a good kind of low friction way to get started doing this. And frankly, when we're talking about looking at HTTP traffic, it's all you need as well.

Are there any security frameworks or best practices that are well known that people should be following?

Yeah. So OWASP is a really good resource. OWASP is the Open Web Application Security Project. They're a very well renowned, not for profit organization that produces a number of different artifacts about web security. And one of the artifacts they produce is called The OWASP Top 10 Web Application Security Risks. And in fact I've got a number of courses on pluralsight about that as well. And that tends to be a very good go to resource for securing your web apps. And they do also have one for mobile too. They've got one specifically around mobile, not necessarily specifically around APIs that probably falls a bit into both categories, but they've got great resources and I find that they tend to be really good reference points where I can send developers or in my normal work, if I get anything outsourced to vendors, I find that that's a really good place to send vendors in terms of these are the security standards that we expect. And hey, it's an open, not for profit organization without a vested interest in any one given technology too. So there's a language that we can all talk.

Is there one thing you see over and over again that engineers designing APIs are doing wrong that expose their APIs to being hacked?

I think the most common one I see is improper SSL implementations. And improper is a bit of a broad term, but it's sort of the things we covered earlier on. So not using it at all is a very improper use, the other one being not validating certificates. So there's a really interesting trick I show in the course where with Fiddler, you can load a certificate onto the device. So, you know, yes, you can effectively compromise your device and you can see all the traffic of apps that are doing it properly, but if you don't load the Fiddler certificate on the device, but you tell Fiddler to intercept SSL traffic, It re signs the traffic with its own certificate. So by the time your device is actually loading this HTTPs address, it's getting a certificate that's not valid. Now the interesting thing there is, does the app then say, hey, this certificate is not valid, you shouldn't be here, or does it just simply process the request and as you would have seen in the course Joe, some apps, even some very prominent apps, just happily process the request because they're not doing validation. And developers do this because it makes life easy. So they're building services there on their local PC. Let's not worry about going out and getting a proper certificate from a certificate authority. We, what we'll do is return our own self signed one which won't be valid on the device. So we'll turn off certificate validation and then they just leave it off. And I've seen banks do this as well. This is not just, you know, some guy in a garage made a bad mistake one day. This is multi billion dollar organizations. So that was actually an interesting one when I found that one. It happens.

I was reading the comments and someone actually mentioned that they implemented some of the recommendations you showed for SSL certification and they found an issue within their application. So the things you talk about are really real world and you can apply them right after.

Yeah, that's it. And look, honestly, in the time we've been talking the 20 minutes or so, anyone listening to this could have easily gone out proxy their data and found a whole bunch of issues in their apps. I mean, not an episode goes by where I don't go, hey, let's have a look at what's happening on the iPad and we'll see. Crazy stuff.

If I have client side validation going on to make sure there's no funky data that's being passed to my API, should I consider my API safe?

So I don't like ever kind of getting to the point where we say something is safe because it's such an absolute kind of statement. And you know, we all sort of draw real world analogies to try and explain technology paradigms, but I think it's a little bit like saying your car is safe. Some of them are pretty good. If I'm riding around in a Mercedes S Class, I'm probably going to be more comfortable than in a, you know, cheap machine from China somewhere. But you know, that doesn't mean that if you drive the thing 100 mile an hour into a brick wall, you're going to be okay. So I think that in terms of things like the client validation it's the same message that we've always had with the web, which is validation on the client is fantastic because it's very responsive and it gives you a lot of immediate feedback, but it needs to be supplemented by validation on the server. And really, you got to do it twice. Some frameworks make it very easy to do it twice, but you've got to do it on the server because the client can be circumvented. And exactly the same logic applies with APIs.

Awesome. That's the reason why I asked the question, because I know in the course that you said even if you're having client side validation going on, you still need to have the server ensure that what it's getting is correct, because there are ways to go around both the server and the client. So you want to make sure you have it on both ends, I believe.

Exactly. And this is one of the reasons why I like using Fiddler a lot as well. And obviously in the course we look at things like the composer feature in Fiddler, where you can reconstruct your own requests, and of course you can always go and do that with Curl or wgetter or anything else that you want that can issue an HTTP request. But when you take the client out of the picture altogether and you take out all of the safety nets that it had for you, things tend to open up a lot more and you see things in a different light. So that's one of the other things I'd suggest. Just look at the HTTP requests and focus on those and forget about the client and see what you find.

Just because API communication occurs under the covers and you don't see it, do you think some developers get a false sense of security, that no one is really going to poke around to find their API's vulnerabilities?

Oh, absolutely. I think developers of all sorts of software get that sense where it's like, hey, no one's going to look at my things, it's all good. But I do think that again, just the nature of API based communication behind rich client apps being that further out of sight makes that scenario worse. And what I try to explain quite frequently is you've got to assume that the attacker owns the device, the attacker owns the connection, and they can manipulate anything at either of those points. And once you start to think that way, you start to think about risks in ways that you wouldn't when you assume that everything is just working the way that you designed it to work.

So what do you think is one of the key takeaways someone could take from this conversation, moving forward with their security testing efforts, sort of the key.

Messages, just to try and encourage people to get in and look at apps. And I'm going to say this generically, not just your own apps, go and look at other people's apps and be nice when you do, because you will find things. But I find that when you start to look around at the broader spectrum of how these apps are operating a, you're probably going to be a bit scared and disappointed and that's maybe not always a bad thing. But you do tend to start picking up on some of the patterns, patterns that you probably don't want to see in your own apps. So go and sit down one afternoon for an hour or so and proxy your traffic and just start opening stuff and see what it does. I don't think it's just me, but it's quite a fascinating thing when you do.

I agree. I mean, just like I said, watching your training video and seeing what's actually being exposed really blew my mind. So I can't wait to start it on my network and take a look at some of our applications, actually see what's being passed back and forth. And I think it's actually going to help start some conversations around security and start questioning things. Like you said, do we really need to expose this? And if not, maybe we should just have a little foresight and take care of it now before it does become an issue.

Yeah. And also ultimately going back to the title of the course, hack your API first, the whole premise of this, and this is actually a follow on from another course called hack yourself first that was more web browser based centric. All of this is about how can you find vulnerabilities in your own things before other people do. And my belief is that unless developers build up the capabilities to actually exploit these risks, it's very hard to defend against them. So yeah, sort of take the time to put your attacker hat on and really try to hack yourself like they would hack you.

Okay, Troy, before we go, is there one piece of actionable advice you can give someone to improve their API testing efforts and let us know the best way to find or contact you.

So the one piece of advice is proxy your device. So proxy your device, run your app, watch what's happening. That's just the number one thing. And everyone can go away and do this right now. And if you're not sure, just Google proxy whatever my device is via Fiddler and you'll find it in a few minutes, contact wise. I am at Troy Hunt on Twitter just troyhunt and my blog is troyhunt.com and most of what we spoke about here is if it's not in my courses, it's in my blog and my blog has links through to all the courses and an email link and all that sort of thing as well. So they're the best ways to get in touch.

Thank you Troy for your security testing awesomeness. For links to everything of value we covered in this episode, head ON over to testguild.com SSLIXsecurity03 and while you're there, make sure to click on the Try It Today link under the Exclusive Sponsor section to learn all about Micro Focus Fortify, which is the recognized market leader in application security. You definitely want to check it out. So that's it for this episode of the Test Guild Security Testing Podcast. I'm Joe and my mission is to help you succeed with creating end to end full stack automation awesomeness. As always, test everything and keep the good. Cheers.

Thanks for listening to the Test Guild Security Podcast. Head on over to testguild.com for full show notes, amazing blog articles and online testing conferences. Make sure to subscribe to join the Guild and continue your testing journey. This has been a Joe Calantonio production.