OpenNSM (Project Meeting) 9/22

Hello everyone, welcome to the, uh, another meeting of the Open Network Security Monitoring Group at the University of Illinois at Urbana-Champaign. And today's meeting will be a project meeting, so we will get started working on the things that we have on our task list, such as the Container NSM project and the DocuMalware container analysis project.

Among others, if anybody's interested in those.

So let's go ahead and get started through the meeting notes here. Get it up on the screen. Go right into the group updates.

And just a tad bit bigger.

So updates on the Amazon AWS credits. We still do not know the amount, but it will be applied in 2 days. So on the 24th, we'll have those credits available. And again, um, We were told around 7,000 to 8,000, but they put in recommendations for 35, I believe it was. It was above 30. So hopefully we get that recommendation.

Hopefully that was accepted. That'd be awesome because that would give us a lot of stuff to play with, a lot of credits to put into infrastructure.

Also, last week I released on the NSM course 2 videos on writing your own Bro analyzer. So this— they're draft videos, so if you want to watch them, please do give me feedback. I know some of the lingo used on Binpack is actually incorrect. I need to do more research on it, but for the most part, we build an analyzer live for the RIP protocol, the Routing Internet Protocol. And I just, I go through the whole thing starting with using Vlad's templates, um, and then moving on to in the second video, uh, how to work that analyzer into data that is represented in Scriptland. So you can actually pull out the version number, for example, or the type of the packet, whether it's a request or reply. In that video also, like I said, I do it live, so you'll see me like going through the RFC, finding out what fields I need to parse out, etc. So do check that out and give us feedback. We'll run along to the meeting sections.

Some awesome questions popped up in chat there.

Okay, did you read them?

I didn't get to see the whole thing. Okay, we'll get to that in just a second.

Networking news. So a few articles that came out. Covert Man on the Side Attack from NetResSec, the guys that do NetworkMiner and other really cool forensic analysis tools on the network. They came out with this article on on basically as a response to what was the NSA's Quantum Insert attack. You can see that they give a little definition of it and why tools like Suricata in the past were not able to detect it based on the way that the sessions were created and torn down. But the idea here, the man on the side, is that, um, for you to revisit the Quantum Insert, there's a request And what happens is, since they are— the NSA and other organizations typically have visibility that other people would not have, they are able to guess the right TCP sequence number and respond before the actual legitimate web server, therefore injecting some sort of payload that the client would then interpret. In the case of HTTP, they could inject a different page or a redirect or something that would cause the client to go to something different rather than what it was intended to go to if they talked to the legitimate server because the NSA could respond very quickly because of the points that they have visibility on in the network. So this basically breaks that down and goes into technical detail— or details of it with the various flags and why Suricata cannot detect it. I have not read this article yet. I was going to today. But I highly recommend checking it out.

I will do so later.

The other article we have up was something we talked about a little bit last week on the malicious backdoors in the Cisco routers. Take a look at that. There was, there was a number of them found on different routers having backdoors, and they have it grouped here by the countries that had the infected routers. You can see that we had the most, and then there's a number of other countries that had less. So there seem to be clandestine attacks potentially related to the SYNFUL NOC router implant that is then discussed in the next article.

But then again, I apologize, I didn't have time to read any of these, so you'll have to check them out for yourself.

But FireEye did a blog post on the SYNFUL NOC or the Cisco router implant. So do check this out and see how they were doing it. Looks pretty technical. I already like it.

TLB read/write attributes.

How they got the malware on there. Analysis of the malware. Yeah, this looks like a really good read.

So if I do read it for the next meeting, I'll try to present on it and just talk a little bit about it, but I haven't had a chance.

Anybody else have a chance to read any of these articles or review them?

No?

Close out of those. So, conference corner. DerbyCon is this week. I believe it starts tomorrow. I will be speaking there on ILLIT. So, if anybody is going to be at DerbyCon, please reach out to me. We can hang out or something. Also, the week after is Ohio Linux Fest, which many of us in this room will be attending. And I will give a talk on ILLIT, and Jon Schipp is going to give a talk on Jekyll, and Mike, who's on remote attendance, is going to give a talk on Vagrant and Ansible provisioning. So we got 3 people from our groups giving talks this year, which is really cool. So do check that out. On the Tool Trade section, Brandon, who's on the remote call right now, actually wrote this tool to help us— to help his team take a look at— basically analyze PCAPs with different tools that work in container.

I was gonna let him talk about that.

Hey, Brandon, do you want me to give you screen sharing? Do you want us to share your screen?

Uh, yeah, that will work fine.

Okay, if you click, uh, share screen, I think it should be a green button.

Yep, I got it.

Cool, you guys can see that?

Yes, can you make the font bigger?

Yep.

How's that?

It looks good, thank you.

Okay, cool. No problem, man. So like Jon was saying, I created this quick little tool to help out some guys that I work with. Really, a lot of times we'll have a pcap that we've been able to collect out of our environment and, you know, either it didn't replay, you know, like it wasn't detected quite right, or maybe we thought a signature should have fired but didn't. And so we found ourselves having to replay PCAPs multiple times, and sometimes it was useful to replay it, you know, if we captured it with Snort in our environment, it was helpful to replay that PCAP through Suricata or through Bro or some of the other NSM tools. And so it's kind of funny, when we started talking about the, the contain-nsm effort, kind of aligned itself pretty well with what I had done. I had had a lot of problems with trying to install multiple versions of Snort as well as very specific versions of Snort dependencies that replicated our production environment, and then putting Suricata on top of that, and sometimes it has same dependencies, and it just became really a complicated to do it within a single VM that I could hand out to multiple people. And so then I started, okay, well, I'll create a bunch of VMs. And now I've got a bunch of VMs to hand out to people and it just wasn't working very well. And, and, uh, so I was down at the, um, BSides Iowa and some guy gave a quick talk about Docker and how awesome it was. And I was like, wow, that really solves this problem where people can just pull down these images. I can create quick little scripts to that kind of wrap themselves around those Docker containers, and away we go. And so this little effort was, was born out of that. So one of the problems that I first wanted to solve was like, it's great to provide a container that has Snort installed on it, but really installing Snort's like, it's a barrier to entry to start playing around with Snort. But once you get policy management and how to create snort config files and how to include all of that stuff was really problematic for a barrier to entry. And so I kind of dealt with that as well. Well, I'll kind of get into the tool. There are some particular directories that are important, but they're pretty self-explanatory. So you put your PCAPs in the PCAPs directory and You can make your own policies in the policies directory, and right now it's based— you have a policy for each IDS engine that you can run right now, only Snort and Suricata. But so if you go into Snort, you can see that I've got multiple rule sets here, and it's the same thing for Suricata. And so you just kind of build out your different policies for your different IDS engines under policy, and you can put lots of pcaps in the pcaps folder. So, the logs directory is the same way, but the logs are populated when the engines actually run. So, we'll do that real quick. So, my buddy who I work with, his name's Philip, he actually just committed this run_engine_python, which is a Python version of my really bad Bash script that does the same thing. But, we'll use his, 'cause he's a cool guy. So all we tell it, you can tell it by default, you can just tell it to run a pcap. And so it goes out and looks for that pcap in the pcaps folder and does Docker and does magic to map all of the, the policies, the pcap, and the logs into the Docker container. So that way you can read all that from the, from the host. So by default, the Python script provides some defaults. It's going to run Snort. And so we do that real quick. Sorry. And so it started up the Docker container here. It's loading up Snort. And so Snort reads the pcap real quick. This test pcap that's provided is actually from Brad Duncan, who does the malware analysis or malware-traffic-analysis.net site. He was— he talked to this group previously, and I just pulled down one of his pcaps and dropped it in there. So I'm pretty sure it does crypto wallet or something. We'll find out. So now that Snort's done, you know, you can roll back through the history here and read what you need to read. Of course, Docker's got its Docker logs that logs all the standard out, but all the IDS alerts are going to go into logs. And so now if we go in here, we can see that we've got Snort, and then we've got a timestamp and the pcap that was written as a— the pcap that was read as a unique identifier that contains those logs for that run of that pcap. So if we come into that, uh, then we can see the actual Snort logs. So, uh, so there's the logs. So you can take that same pcap, run it across Suricata, and it does the same thing. Um, that's more or less it.

Oh, very cool. Thanks for sharing that. So yeah, when I was— whenever I was thinking about doing the Kadena in a Sample, I really wasn't even thinking about like all the configuration possibilities. I think since you reached out to me, kind of opened my mind up a little to say, wow, we could really make this even more, um, personal than just a training. Because training was my original idea, just to let people to pull down a tool, try it out, like a student for example who has no prior knowledge touching the tools, then kind of go over the output with them. But for analysts and stuff, you know, if we can have— we can actually create separate different policies that are common with different settings and stuff like you have in the policies directory and write like a front end, like your run engine, we can actually do this very well across all the different tools and check out all kinds of analysis. I think that's actually very powerful.

So yeah.

I think, yeah, this is definitely a motivator, I think, to just kind of explore all the ways that we can run these tools.

Yeah, and you know, from an analyst's perspective, there's no one tool that will give you all the answers that you want to ask while doing an investigation. And so, you know, there are many cases where I want to just parse out all of the DNS requests, and Suricata or Bro are really great for that. But there are other cases where I don't need to ask that question. There are other ones where I just want to see all of the HTTP requests in this, and Tshark's really good for that. So yeah, it'd be awesome to be able to build kind of a wrapper around all of those tools so that way you can say, go do this.

Yeah, very cool. Yeah, I think we're going to work on that though.

Cool.

Thanks for sharing that, man.

No problem.

Okay, so that concludes the notes we have for today. So today is again a project meeting. So what we're going to do is I'm going to stop talking unless there's questions, and then we're just going to start working on things. So we have two projects again that we are, I guess, kind of prioritizing. That we can do in a classroom setting. That is ContainSM, and that is Docu, the malware container analysis. So for everybody that's on the call, if you need help getting started with one, do ask in chat or just unmute your mic and ask, and I can help you guide you through getting started. If you have just wanted to want to have your own discussions, feel free to talk among yourselves or in the chat, and we'll just kind of get started. I'm going to begin working on a new sort image to test Mike's out, trying to apply a new version. And then I'm going to work myself, then I'm going to, after I do that, whether, you know, 10 minutes into that, I'm going to start working on the malware container project and start to look at the Docker API.

Hey, Chip. Yes?

I'm actually already on that.

Say again?

I'm actually already on getting a new container version up and running.

You said— I'm sorry, I still couldn't hear you.

I'm already on that process, getting the new container up and running.

Excellent, excellent. Let me know how it goes. I'll let you do that then if you just want to work on that, and I'll, um, work on something else like the container.

Yeah, so finding versions older than, uh, two older versions might be an issue. It looks like they don't host, uh, older versions on their website. Yeah, SourceForge only has the last two, so—

Yeah, that annoys me so much. I really wish they put it all in a directory list and then you can just look them down.

Awesome, I'll get these two down and I'll look some more.

Yeah, I know, I wonder how many SourceForge has, but I hate downloading from there, their URL structure blows.

And their documentation says they only have the last oldest version.

Okay.

It looks like I may be able to find two on there, so. We'll see what happens. I'll get these done and I'll look further.

All right, sounds good. I'm gonna hit one of the lights in the room and then, uh, kind of do our own thing. And feel free if anybody wants to talk about anything that's not related to Golan.

We just—

this is just free time to work on stuff. Need help with Docker? If you need any tools related to that, let me know. We'll get you started.

Jon, is there a Trello roadmap for that other project?

For continuous SAM?

Yes.

Nope, not yet.

I did put a link to the one that we're using for Daku in the Slack channel, Daku. Okay. There's the link there. And if you sign up for Trello, if you're interested in helping out with that project.

Can you guys hear Mike or Shane talking? So Shane said that he put the, uh, the link to the Trello inside the Doku Um, chatter on the, or on the Docker channel on our Slack group. Is that correct, Shane?

Yes.

And there it is right there. You click on that link and then that will show you the roadmap. So basically we're using Trello, um, which is recommended by Jon Schipp sitting next to me, to actually kind of organize and show what we're working on. So you can drag a board here if you, if you know that you're ready to create the server environment, then that's in progress. You can move into different things. If you're blocking on something, like I'm working with another person, you move to that. Bug reports, Q&A, and then done. It's just a way to do project management.

It should be noted that the parentheses numbers— we're using a, uh, we're using a plugin which we can provide the link for, uh, allows you to document the estimated hours and the completed hours. Uh, the goal of course is we're trying to determine the, uh, We're trying to determine the velocity that we're doing this, and it actually generates, uh, burn-down charts and everything if you have the plugin installed.

I don't know if I have access to adding people to the team anymore now that this is done.

Are you a certified Scrum Master?

No, I listened to an audiobook on it though, so—

yeah.

Do you have to like pay money to be Scrum Master?

Apparently, yeah. There's like companies dedicated to, uh, for some reason, yes, certified teachers.

Shane, you're an admin now.

Okay, yeah, we're going to try to add, um, if anybody else wants to let us know, we'll get you admin access.

All right, so I'm going to get started. I may turn off my screen sharing. I don't know yet.

We'll see how it goes. So, I'm going to python docker.py. I'm going to start with the stats docker. Oh, Vlad said it's private, so we need to make this easy for people to find and add themselves to.

Yeah, how do you make it public? What's that?

The board. Oh, the board's private? Yeah.

Yes.

Uh, which one, the Doku?

Yes.

Um, so look, now you see where it says, well, you see where it says team visible?

Yes.

And now below, if you can switch that right there.

Ah, perfect.

Alright, it's now public.

Why am I signed up to nothing? Wouldn't you still need to be signed up to get added to the workflow? Let me post that in the, um, if you want to actually help out.

Ah, hey, uh, Mike, did you see that link that Brandon posted in the channel about the Snort old stuff that you shouldn't use?

Yeah, I replied to it.

Okay, sorry, I'm just now checking the chat, so no problem. I'm gonna stop sharing my screen so I stop jumping around, and I'll set the, um Yeah, movie. Okay, well, it seems that there's the API client, it is docker-py.

That's the one done by the Docker people. For those working on that, post it in chat. Hold that down, check it out.

But it's obnoxious.

He's one of us. He's one of us.

Why are you staring at me?

Let's picture this. All right, so just everybody knows, I'm working on the, uh, container creation code. Trying to figure out how to do that.

Ah, dang it, Devin did not put his, uh, he did, did not.

If you need something to work on, let me know.

See, you see that references and resources, that's where I should go to.

Okay.

So one of us can work on parsing YAML files or JSON. So we have a config file and just kind of get it working.

Yeah, in Python. So it's like, yep, be able to name equals container name, you know, CPU equals— so all the common Docker settings to be able to pull that in.

From the file.

You might be able to already do that with the API.

I haven't tested it.

So, or if you want, we really need like the, starting the design of the web stuff.

Thanks.

So, so what's the difference between this and using this one?

Well, we have it.

You see this page? Yes. So with that, we just make HTTP—

well, hold on, I think the—

there was an HTTP request, right? So the idea is you have a command line tool or something, are there— whether it makes a backend request to create the container.

And it makes the request to create it.

Okay, so it does all the work, but the web interface would do— um, yeah, the web interface is going to expose an API that's simple, like org.com/create/container name, right? So just, just for example. And then on the back end, there would something be the in-bottle, match that request, and then it actually made the proper request for the Docker API. I am a little confused now because I don't know if we should use this one or that one, or what is the difference. So I'm going to try to investigate that. So, um, I guess we don't need to have multiple tickets, we just need to have a So the idea is to have a command language tool and then a— whether a subject will be— you know, right, right. And that'll be presented in the web interface. The command line tool would have to be too. So is there a way to get both tools to use the same thing, just different— what is the best way to do that solution?

So I was just thinking like there's like a unified API. Command line tool makes a request to this.

Look right here.

Command line tool makes a request to this.

This actually does Docker container creation. Web interface makes a request to this. This actually does Docker creation.

So there's just—

all you really need is the central piece.

That's kind of how I see it.

I don't know if you have a better idea, but that's kind of how I see it.

Um, you do be able all the same things you can do.

You want to be able to make it so analysts can use it directly. Yeah, so the middleware, what I would call the— all the work is done, is we'll have access to the directory. I'll know that either of these config files, I need to monitor them for new ones. And when the client makes the request to that middle piece, the middle piece returns the latest updates of things. There's a new image added for XYZ's SA tool. Can you send that to me?

You don't have to do it right now. I'm just saying yes.

All right, so we have this, right? And then over here we're gonna have Doc.

So let's see, so, uh, we're gonna—

so how do you want to do this? Clem and I— I wonder if my mic was on, because—

Clem and I are working up a diagram for how this should work. So apparently that needs to be done first, clearly.

Um, diagram of what now?

How the system should look, the architecture.

Architecture of what?

I think I missed that.

The architecture of the Doku software.

Okay, gotcha.

Thanks. We just like getting ready just to start right now over here. We're like, wait a second, some things, some things aren't making sense already.

So he's over here designing it, and we'll share with the group when it's done. Thanks.

Is anybody working on our projects?

Just curious if anyone else is working on anything specific.

We're learning how to build Docker images, but that's not very specific.

Oh, I mean, that's a good start.

I talk about boxes and that just wasn't really working. Okay. So this is the status.

The longest process of this is actually building the Snort containers to test.

Say again?

I was saying the longest part of this is actually building the containers to test. Takes quite a few minutes to build a Snort container.

Right. Well, I think it could be both because we're going to need containers for this one.

Were you specifically talking about the project, the continuous SAM, or are you talking about images for both of them.

I was talking about the Snort containers that I'm building. Oh, I think if they were to build them just to test, I mean, changing the files and taking them at all, I want to test them all before I commit it.

So in our malware tool, but, um, because of the example requests, you want to be able to see what signatures match.

Good enough.

Yeah, but yeah. Um, yeah, Mike, whenever we have the infrastructure, we can dish out like hopefully 16-core machines to each other so that we can start cranking on building those.

That'd be wonderful.

Yeah, I know, I did it so many of them, it took me forever on my laptop. All right. You just did the script, didn't you? Yeah, I did.

This is—

okay, here's—

and this is—

okay, this is an R function, right? So, um, This is your return. Yeah. So you were using Alright, so one thing that's interesting, I don't remember systemd being installed on this, it might be a mistake.

Which is not a problem, it's just I'm not too familiar with it.

So let's do this again, what were the errors that claimed that key, so let's actually remove it and hope that maybe Docker will regenerate it. You like that?

Actually, actually, this is pretty sweet website, ASCII Flow. Yeah, A-S-S-K-E-Y F-L-O-U-W. See what I did there?

That's what I was trying to say in words. You got it.

So, have we—

what do you guys know? Have we defined like the way we're going to interact with this thing? Or does my question make any sense, as they very rarely do?

By interaction, you mean the users interact with it?

Yeah, well, and then I'm including like the config file stuff in there. So I'm assuming though, after we've So, if we skip a couple steps there, we've already defined the Docker images and we've somehow set up all the Docker containers. Once we're trying to run stuff inside of a container, would there need to be instructions for that other than are we just going to do a docker cp and put it inside the container and then execute it inside the container?

That's what I was thinking.

So you don't see any use cases where it may be like, well, instead we're gonna do something other than that? I'm just trying to think of like what the config file should look like, or—

So I think the config, I think, will be separate. I was thinking that the user would be an option in the—

so you're on the command line, so we'll say it's docu, docu --xsource or --malware, and you specify the file name of the malware, and then use whatever options are on your various images. It will read those configs, the API will spawn up those containers, it will copy that malware file you pass as the source, right, or the file to each container. It will run them in parallel currently, and then it'll be able to capture the results and store it somewhere for the actual review, essentially. So that would be—

oh, go ahead.

Does anybody have any free hands right now that has experience with VirtualBox?

Has anybody used our Vagrant config on Windows yet?

Okay. Does VirtualBox work well on Mac? Sounds cool. I've never had a problem. It's really bad on everything else. It's a little weird. I've used it on Mac a little bit, but it's just hard enough to tell it to do what you want it to do. We've been using it a lot, and I've noticed that 8.1 and 9.10 Windows especially is just garbage. It's more of a garbage pool. It works. Really slow compared to VMware. And it still works. Yeah, VMware is the order of the day now. I would probably stick with VirtualBox. VMware wasn't operating for us. I can't even— yeah, I can't even— I think I found VMware best in here.

What's that?

Okay, so what we're gonna do is using this clip. I'm hoping this one— I don't honestly have no idea how, how long they were combined.

Yeah, unless I did not test it.

That's possible, but I made a mistake.

So, go back, actually, do it from the command line. Do you have the directory where you typed vagrant up in?

Or that window?

So, open up your console. You just need a shell. I mean, well, command prompt essentially. And then go in that directory and type vagrant destroy. Because that will, um, Okay, so it's committed.

If you're on Mac though, OS X, I highly recommend doing the Docker—

what was the new name for it?

Uh, let me look.

Toolbox.

Yeah, Docker Toolbox. That's what I'm using.

So you can run it directly, the Docker client, right when you're up on your OS X.

If you just Google Docker and OS X, it should come up with the JSON.

So is there something new from Docker?

Yeah, I think it's kind of, I think it's considered sort of more recent. I'm admittedly still figuring it out, but it seems that it's— I was able to get stuff up and running without having to use a separate VirtualBox.

I believe that VirtualBox might sort of—

So when you do port mapping, does it actually map to your Mac? Because I know that's one thing that always tripped me up on Hoops and Docker.

What happens is it spins up a VM in the background with Vagrant and you don't even know about it.

You can hit Alt+F3 and like, whatever.

So, it is Vagrant LVM?

Yes. Oh, okay.

Yeah. So, it does use VirtualBox.

Yeah, so if you map the host port, right, it's not actually mapping to your Mac, uh, the Mac host. Ports, right?

That makes any sense?

Or for what? Can't brain right now. So like, one of the things that always messes up with Docker, right, if you're mapping ports, right, that instead there was that VirtualBox shenanigans in the middle there. So like, you couldn't say you can access like your Mac's IP address or its actual network IP address or whatever.

Right.

It's still that way, right?

You can modify the bigger file. I don't know where it is, but you guys can access the original.

What's the difference between the Docker Toolbox?

Toolbox superseded Boot2Docker. So Boot2Docker is considered old, and they redo the whole thing with Toolbox. Honestly, I think it was a marketing thing just to make it sound better.

Docker Toolbox.

But Docker Toolbox consists of Vagrant, VirtualBox, the Docker client, and some other stuff.

All together, everything like Docker.

Yeah, pretty much everything is in there.

Yeah, no, they didn't.

Now it's just more shenanigans.

Yes. Yeah. All right. So now— What's that? Yes.

You look delicious.

Get in my belly.

Okay, put on the diagram.

Oh, it's been done.

It's been done.

We've moved on to describing.

Hey guys, it's Brandon.

Thanks for having me. I'm gonna head out.

Okay, thanks.

Take care, guys.

So, uh, this is what Clem came up with here. Um, so pretty. ASCII diagrams we have. So we'll have some sort of—

it's a JPEG of an ASCII diagram. Oh, it's a PNG of an ASCII diagram.

So we have this Docker middleware that the command line tool and the web interface will talk to. This middleware will then read from the configuration files up here that tells it what images to load to make MetaContainer for analysis. And then Docker middleware will then send those results or make the request to Docker to create containers of the images specified with the options for these configurations. So, this simplifies it a little bit, I think.

This is tentative, by the way, too. Okay. I like it.

I like it.

Yeah.

If anybody sees any design problems with this, let me know.

Um, so we'll wait on people.

Oh, that's okay. Um, because it's, it's like, it's not like— because I actually wanted to be able to get to do operational stuff.

I worked with Adam.