Metasploit Pen Testing with Keith Watson

Five, four, three, two, one. Drop that. Welcome to the Test Guild Security Podcast. We're all getting together to learn more about security testing with your host, Joe Calantonio.

Hey, it's Joe. And welcome to another episode of the Test Guild Security Podcast. Since the show started, I've already gotten a few requests asking me to do a show on penetration testing using metasploit. So I actually interviewed Keith Watson, the author of the pluralsight course Penetration Testing using Metasploit a few years ago, and I thought I'd share that with you today because the content is still awesome. So Keith started his career in information security way back in 1997 at sun, developing security tools for the enterprise. He then went to work for Purdue University as a research engineer with the center for Education and Research, Information Assurance and Security. And this is when I interviewed Keith. But since then, Keith has actually moved on and now works at Optive Security as a core services architect, assisting clients with their security needs. He also leads an active local security interest group, which he started in 2009. As I mentioned, when I did this interview, Keith was working at Purdue University, so that's what we talk about most. But all the content here is relevant today for your security metasploit penetration needs. You don't want to miss this episode. Check it out. It's no secret that the number and complexity of applications are growing, but what is alarming is that 90% of security incidents result from exploits against defects in software. The need for better application security has never been higher, and Micro Focus Fortify is here to help. Fortify is the recognized market leader in application security, and it is the most comprehensive and scalable application security solution that works with your current development tools and processes. With Fortify, you could start security applications in a single day, including custom code, open source or commercial components, and scale as your needs grow with an on premise as a service or hybrid implementation. Check them out. Head on over to micro focus.com app security. Hey Keith, welcome to the Guild.

Hi Joe, Glad to be here.

Awesome. So, Keith, just curious to know, to start off at Purdue University, what kind of security Metasploit penetration type testing do you do?

Well, it's a variety of things. I mean, we're a very large employer. At the West Lafayette campus, for example, we have 12,000 employees, we have 40,000 students. And if you add in all our regional campuses with their students and their faculty and staff, we're looking at about 92,000 people. The university itself has 300 buildings and each building has WI fi in it. So we have a variety of threats because we have a fairly open network like most universities. We want to encourage academic pursuits. We want to make it possible for faculty to collaborate with other faculty at other universities. So our network is very much unlike a corporate network. We do have a lot of intellectual property, certainly we do have a lot of research going on. We have to balance, and it's a tricky balance. We have to balance having an open network and really facilitating a lot of collaboration and protecting people and systems and data behind there. And it's a challenge. It definitely is. There are a lot of things that you could easily implement or enforce in a corporate setting or even a government agency, for example. We just don't do it. And that's mostly because we want to maintain the mission of the university and really focus on providing open access wherever we can.

Awesome. I guess it really highlights that there's probably a lot of situations that people don't think they may need this level of security or security testing. But in fact they probably do. And the university, like you said, is a great example. So why do you think penetration testing is an important aspect of security testing activities that folks should be doing at most companies?

I think penetration testing as a practice has been around a long time and in fact it is required for a lot of regulations. So, for example, if you take credit cards, you're subject to PCI dss, which is the data security standard. And there's a requirement in there to conduct penetration testing. Now, most organizations either have a penetration testing team and they use them to, you know, maybe as part of a security analyst job is to go and do penetration testing within their own organization. But a lot of times we see organizations that will hire a consulting company or specialists in penetration testing to do that work. Especially in the PCI space. There are a lot of vendors that focus totally in pci. They provide architecture services, they outsource some of the security monitoring, and. And they also do penetration testing as part of the services that they offer.

All right, cool. So let's get into Metasploit at a really high level. What is metasploit?

So metasploit is a really fantastic open source framework for conducting penetration tests. And not only does it help you with conducting the test, it also has auxiliary modules that can help you scan a target system looking for vulnerabilities or at least system configuration, or what we typically call reconnaissance. Right. We're trying to define more about what that target system has. It also has a number of post exploitation capabilities. And that really is once you penetrated A system where you found a weakness that you can exploit and you gained access to that system. There are a lot of tools such as MeterPreter, which allow you to do further testing of the system to what we call pivot or laterally move to another system also on that target network. And this metasploit is really the framework that ties all that together. It has basically the open source version and I'll talk a little bit about that and the commercial offerings. The open source version has a great command line interface and so you really have to know how it works internally to understand how to use it effectively. And then there are open source options for an API, a graphical user interface, which gives you the ability to kind of simplify some of the tasks that you would do when you're targeting specific systems. So I mentioned their commercial offerings. The company Rapid7 basically acquired the lead developer of Metasploit HD more several years ago now. And they continue to offer it as an open source solution. In fact, they have some full time engineers dedicated to expanding and building in capabilities to metasploit. They also offer a couple commercial packages which extend the capabilities of the tool or offer different interfaces. So there's one that is. The name escapes me. I think it's metasploit Express and it has a web interface to that command line interface that most of the open source people would use. And then there's an enterprise version which provides integration with some of the other Rapid7 commercial tools. So basically the open source one is where I have encouraged most people to start. Because while you can use the graphical user interface to simplify some of what you do, you need to understand how it works under the hood. And you really get a good taste for that with the command line interface. And that's really what the course is geared around.

Awesome. You break everything down really step by step in your course, from installing all the way to conducting a successful penetration test. So it seems like a very powerful tool. And for someone that's a beginner, say someone's more into testing functional testing, but they want to get more into security testing. What type of skills do you think they would need in order to really make that transition into using a tool like metasploit?

I think that the challenge here is that the way we use metasploit is really about finding and exploiting vulnerabilities and you need to kind of have the skills to think in that, that mindset. So curiosity is certainly a key aspect of that. And being able to effectively use the tool is another most of the included exploits that come with metasploit exist in or exist to exploit known vulnerabilities. You can build your own exploit capabilities. So you can say, discover, I know there's a vulnerability in this particular tool. Metasploit has the framework and basically it's Ruby code under the hood. You can write modules that allow you to take advantage of that vulnerability and exploit it using the framework. And it simplifies a lot of the overhead that you would normally have to do if you were writing a standalone tool. For example, there's a lot of built in capabilities in the library that allow you to simply spend a few minutes developing your exploit and then putting it into the framework and launching it and then doing all your post exploitation capabilities. So skillset wise you really need to know Ruby programming, but Ruby's fairly easy to know and some of the capabilities of the metasploit library to take advantage of that. Probably the key aspect to penetration testing is really the reconnaissance side. It's really finding about what the target is, what services are offered, what vulnerabilities it may have if you suspect it has some, and trying to trigger some vulnerability so that you can develop an exploit module around that, for example, and that would allow you to just plug that all into the framework and take advantage of its capabilities.

The real trick, and I jump right to this, is being able to understand when and how to use a tool like metasploit. So like I mentioned, there's a lot of exploits built into the tool that come with it. They get updated all the time. And as a security professional, we know that, you know, when we're using it, we operate under certain codes of conduct. For example, we want to make sure that we have permission to target a particular system and use a tool like metasploit to look for vulnerabilities and possibly take advantage of them. And I think that some people, and in fact I had an interview with a student recently, we were looking for hiring student employees who basically admitted to me that he's used tools like metasploit against the university systems where I come from. That's a big no no. It shows a lack of moral reasoning, understanding the right way to use a tool like metasploit, because there are vulnerabilities out there in corporate networks, even in your house, behind your little home router, there are vulnerabilities that metasploit can take advantage of. And really you need to have the right understanding of how and when to use that tool and having the right permission to do that sort of testing, not everybody has that. But you may have seen in the first module of the course, we jump right into ethics and we talk a lot about how to use, how to use tools like this and when to use them.

Absolutely. And that's what I love about it. You normally don't see that in a lot of courses, but you give the background and the philosophy why, you know, why ethics is important, especially in this situation. A lot of people don't normally think of ethics, but especially in this type of testing, you really have to be careful. So could we just go over, you know, what are the ethics or maybe some high level code of conduct you think someone should follow when they're doing a penetration test?

Sure. Well, let's start with the capabilities of metasploit. They're significant in that if you're using it for good, you're going to be able to identify weaknesses in a target system. You'll be able to use it to inform system owners that there's a problem and we need to fix it. And you can point out specifically why it is. You can highlight changes that need to be made to that system. You can use it to protect information at a high level. Right. Because you're, you're using it to identify weaknesses and basically you're protecting the organization. And if you use it correctly, you're strengthening the profession. When. And what I mean by that is proper use of a, of a powerful tool used properly by professionals, you provide confidence in what we do as security professionals. Right. Now, on the other side of that, the power of Metasploit, again, can be used for evil purposes. We'll just, we'll just leave it at that. Right. It identifies weaknesses in the target systems, which we did on the good side as well. But instead of forming the informing the system owner about issues, you're informing the attackers. Right. You're also highlighting targets that can be exploited. And you're using it not to protect information, but possibly compromise information, which then compromises the organization. And that when you're using security professional tools in that manner, you kind of weaken the profession, so to speak. So use of these security tools for not so good purposes, but definitely casts a shadow over what we try to do as a security professional. And so that there's a balance there. Right. And that's not always obvious. So one of the things we try to tell people to do is to make sure you know what you're doing. Don't just be a script kitty, which is the term for somebody who just downloads a tool and runs it against whatever site they have a beef with, for example. But to know how to use a tool like Metasploit, how to use nmap, how to use other tools like that, you want to make sure that anything you find you're disclosing to the right individual. So usually that would be the system owner or management, or if you're under a contract situation, you'll have spelled out the scope and communication paths. So if you find something or you're providing a report at the end about issues you discovered, and certainly you want to operate professionally, you don't want to be a good, solid, upright citizen during your day job and then go home at night and use the same tool to attack sites you have an issue with. Right. And finally, we want to use this as a way to advance the security profession. And that's really about education. Right. You're educating others about issues you may have found through your own research, or you might be teaching other people how to use tools like metasploit.

Awesome, Great points. I definitely agree.

It looks poorly when you, when you just use a tool for the worst case scenarios or just to do pure. No, not being constructive with it.

Exactly. And something along those lines. They are specifically spelled out in codes of ethics. So for example, the EC Council, which is really the certification body for certified ethical hacking, they have a whole series of code of ethics which expands all the way out to 17 different points. And I refer to that in the course. There's also the ISE Squared, which has a code of ethics which is a little more simply stated. And basically it's, hey, we're here to protect society and the infrastructure. We want to be diligent and competent people and provide that service to the people we work with. And there's other canons to it, such as acting honorably, honestly, justly, that sort of thing. So really those are kind of the, I won't say legal requirements, but they are certainly the code by which we want to operate. And where that gets into trouble is we have the terms of what hat you're wearing. Right. You may have heard the term black hat, white hat, gray hat, that sort of thing. Again, it goes back to the old Hollywood movies of who wore what hat at what time, whether they were a good guy or a bad guy. And certainly we want to operate as a good guy and put our white hat on.

Awesome. So you did mention earlier about modules that are available to you within Metasploit, and I think there's like 3,000 plus modules in Metasploit, so Are there any modules you use all the time or you think someone learning metasploit should start with?

Certainly the auxiliary modules for scanning are quite useful. There are some port scanning modules and port scanning is really about finding what network port ports exist on a target system. So that way you can define what services it may have. So if you know port 80 is open, it's probably got a web server. If it's got ports 135,139 and 445, it's probably a Windows machine. And it's got file sharing enabled or printer and file sharing enabled. So some of the scanning modules are quite helpful. One that's one that's interesting and a lot of fun. We use it in the demo is trying to think of the exact number of it. But it's basically an exploit, an old exploit in Windows systems on file sharing. And basically it's a buffer overflow. And that module basically is pretty consistent in giving you shell access. So if you have some older Windows machines that just haven't been patched in a long time, that's a great module to get in and really just start using the post exploitation tools. So the one thing that I talk about in the course and it's really part of anytime you're doing a penetration testing exercise, metasploit isn't necessarily the first tool you go to when you're getting ready to start. You may have to gather information, do reconnaissance. You may be targeting specific systems and you may want to focus on, hey, let's start with what network services exist or can I even get to the network? And one of the great tools that's not part of Meta, not part of metasploit, but a certainly useful tool is nmap. It really is the network mapper. And it's been around for a long time, probably 20 years, and it just keeps expanding and getting better. It has scripting capability as well. You can find a lot of vulnerabilities that you would find in metasploit. You can find using NMAP as well. And it's really integrated really well with metasploit. So you can actually run NMAP from inside the metasploit command line. And any findings it has, it incorporates it into the database automatically. So there's great integration there. But the point is you're trying to find the services and NMAP is just one of the tools you could do to do that. So you're going to start with the reconnaissance side first and try to say, okay, what does the system have or what Networks or what systems do I want to target? It kind of depends on the scope of your engagement or whatever you're trying to accomplish. And you might rely on other tools that are very specific. So if you know that the target has SMB and it's sharing files, you might run NMAP and you might look at the SMB scripts to see what it can do. You might use the metasploit auxiliary modules for SMB scanning to see if it's got any known vulnerabilities. You might also use other tools and you might even use packet capture and use a tool like wireshark where you're actually looking at the actual packets exchanged on the network to try to see what's going on and possibly use that to see what services are not obvious or what communication is happening on the network that you might be able to take advantage of.

So I'm just thinking in a day to day life of a developer, a lot of times they focus on functional testing, performance testing, and a lot of times they do things like continuous integration where they put it in a. They build and they run every night. Is there any benefit or any uses of using a tool like metasploit that runs every night against the software using, you know, running just common exploits just to make sure that no new holes or vulnerabilities have been introduced into the code? I'm not sure if that makes any sense, but sure.

So let's start with the idea of penetration testing. And it's really kind of the high level concept. Penetration testing is typically done after a system is deployed and all the security controls have been wrapped around it to protect it. Typically that's when we bring in penetration testing. It's like way at the end of the whole testing cycle. Right. And the reason for that is we may have done really great planning and done a great job at architecture and implementation, and we've done testing all along the way and we put it into production. Everything's working great. We've got security analysts and security tools to monitor the capabilities of the system and see if anybody's trying to attack it. But we may have forgotten something. And that's really where penetration testing comes in. Because as I talked about having that in that curious mind to say, if I'm thinking like an attacker, I'm not going to go through the front door, I'm going to go through the side window or I'm going to go in the back door, I'm going to look for ways that maybe the developers didn't think of, maybe the System administrators didn't think of that. The security analysts aren't watching and there are no security tools monitoring that aspect of it. So for example, a great one is I have a dmz. I put our web service behind it. We've got firewalls out to wazoo, we've got intrusion detection systems running on everything. But I have a backup network and the system admins who don't work directly with the security admins, for example, they don't work with the developers, they got to work independently. They have opened up a second network to allow backups to run. And let's say they don't have a firewall on that backup network. They have communication path that allows the backup servers to talk to the servers in the DMZ and they just go through the firewall and no, no sweat, right? I as an attacker might discover that network and I would use that as the way in to the system and not through the front door which has all the security controls around it. So that's where we really want to find those weaknesses that we didn't think of, we didn't anticipate or somebody changed something and nobody else knew about it. So going back to your question about where's this in the, in the software development lifecycle really comes at the end. However, it is possible, like in a regression testing mode where you've had a vulnerability, you fixed it, and then you could use metasploit in a way to automatically test to make sure you haven't reintroduced that or reintroduced that vulnerability to the code. So that's one way you could use it. And you can certainly script metasploit to do that.

So you did speak earlier how metasploit may not necessarily be the first thing you use when you have an engagement. I don't know where I read this, but I heard that the biggest security risk usually is always the human. So when you do an engagement, are you actually at that level where you try social engineering someone to see if you can to get into information or do you just go right to a tool or how advanced are these engagements?

And I should state, I am not a penetration tester by profession. I have done a little work in that area. But typically what happens is, is in the engagement when you're scoping out the project, for example, I'm talking more of a professional engagement here. So if I'm a consulting company, I'm going to scope out what the target is or what the goal is in any penetration testing engagement that might be. Well, I just want to get access to some set of restricted data that I shouldn't have access to, or I might want to compromise a particular host on a network and make sure I can document that I actually did that. There might be any number of rules of engagement here. It might be very broad in that just get in. Right. And that might be finding any way through the network, might be through backdoors, it might be physically getting into a facility and connecting to a network. It really spans a gamut in terms of what an engagement might consist of. So that goes back to your question on how are the humans involved. If the scope of the engagement says, yes, you can do social engineering, then I'm picking up the phone and I'm telling them I'm from the help desk and there's a problem with their account. And if they could just help me by telling me what their username is, their workstation name is, and their password, then I can certainly help them fix that problem that they didn't know they had. So, yes, sometimes the target is information, but the method you get there is through the humans.

I would highly recommend your course to anyone that's trying to learn Metasploit, but also you go over high level ethics things that you would need for any sort of ethics, security type testing. So there are modules there that people can actually pick and choose for what they need. But are there any other courses or any other resources you recommend? Everyone should check out that's trying to learn more about penetration testing.

So for penetration testing, certainly pluralsight has a great series on certified Ethical Hacking, which is really part of the CEH certification. And Dale Meredith and Troy Hunt have done a really great job with those courses. And there's a lot there. I try to remember. I think there's. There's more than 10 courses in that series. That certainly gives you that. That great understanding of penetration testing and the processes behind it and the different techniques used to do it. There's also some tool discussions, there's courses on nmap. And I'm trying to. I'm drawing a blank on the gentleman who wrote that one, Kirk Marshall I think is his name. And it's also in the pluralsight library. And that's a great course. I referenced that in my course as well. And then any course on wireshark is a good one. Wireshark is an amazing tool and, and every time I play with it, I just have a great time, which is really sad, but I'll say that. And outside of pluralsight, there are lots of courses. If you're more into the security profession. Sans has a lot of courses that would be very beneficial. There are lots of YouTube videos to show you different aspects of how to use Metasploit, nmap, Network Miner, any number of other tools that we use. And I really can't say that we use only one tool. Certainly there are a lot of commercial tools where they tell you, oh, it's the only tool you'll ever need. But really on the open source side, there are just a ton of tools to use and they each have their unique capabilities. Some of them may not have been updated in a few years, but honestly they still work great. And there's just a wide ranging set of tools out there that, that any anybody could pick up and use and certainly build a lab at home. You know, if you got an old copy of Windows XP sitting around, throw it on a VM and hit it with Metasploit. You'll have a great time.

Yeah, I was going to ask like, what's the easiest way to get started? It almost sounds like Kali Linux is the way to start.

Yeah, Kali Linux is something that we mentioned specifically in the course. And Kali is a great collection of security tools and they're always being updated. The base OS is being updated as well. So it's not something that you have to install once and then, and then you know, it stays static. It's, it's changing all the time and it has a great collection of tools. So a lot of the things like NMAP and Metasploit, they're just built in and updated all the time. The one exception that I ran into with Kali is that I talk about a tool called openvas, which is a vulnerability assessment tool. And that tool unfortunately got dropped in the update to Kali when they started using rolling upgrades. You can still install it, it just is not there by default. And by the time the course went out, that part I did not notice, unfortunately. But Kali is a great tool.

If you want to build other tools more related to intrusion detection and system monitoring. There's another one, Linux distribution like Kali called Security Onion. So we have a great plethora of tools, but really, if you're looking more on the penetration testing side or even just doing network vulnerability assessments, Kali is really the way to go.

Okay, Keith, before we go, is there one piece of actual advice you can give someone to improve their Metasploit penetration testing efforts and let us know the best way to find or contact you?

Well, you know, I don't want to go and tell everybody go see my course, but really it is a, it is a great introduction to the concepts in penetration testing and it actually gives you some hands on skills with the metasploit tool, the NMAP tool, Open vas. There are probably some other courses out there. None really come to mind where we've kind of brought all that together.

So really when we, when I worked with the pluralsight editors to design this course, we wanted to make sure that we covered something from a beginner level, which is what the course is listed as, and that talked about the concepts in penetration testing that provided real demos that somebody could actually use and had fairly good graphics to show somebody who may not be completely familiar with how exploits work to have those clearly graphically described. And then finally, really the focus was, was on the ethics aspect of it. We really didn't want to have a course where somebody just got access to a tool and then felt like, hey, I've got a hammer now everything looks like a nail, right? How you use the tool is. And where you use the tool is very important. And so that I definitely wanted to cover in the course and that was certainly supported by the pluralsight editors when we outlined it. And certainly there are more hands on tools available. In fact, if you're interested in competition, there's capture the flag competitions where you can actually use a lot of these tools we talked about to actually go and accomplish a goal in a game like setting. So that's a lot of interesting work going on in that area, really. It builds interest in the profession as well.

Thank you Keith for your security testing awesomeness. The links to everything of value we covered in this episode and head on over to testguild.com S07 and while you're there, make sure to click on the Try Today link under the exclusive Sponsor section to learn all about Micro Focus Fortify, which is the recognized market leader in application security. You definitely want to check it out. So that's it for this episode of the Test Guild Security Podcast. I'm Joe and my mission is to help you succeed with creating end to end full stack automation awesomeness. As always, test everything and keep the good. Cheers. Thanks for listening to the Test Guild Security podcast. Head on over to test guild.com for full show notes, amazing blog articles and online testing conferences. Make sure to subscribe to join the Guild and continue your testing journey. This has been a Joe Calantonio production.