The Netsniff Ng Toolkit Jon Schipp DerbyCon 3

Hello. So I hope everybody's having a good time. Can you hear me? Okay. Can everybody hear me alright? Sometimes I talk too loud, but I think the sound's good now. So today my goal is to persuade everyone to take a look at the NetSniffNG toolkit, and we're going to go through a bunch of examples and some features of the toolkit. And hopefully we have some developers in the crowd. I'm not so much of a developer, but I am trying to learn various things like C. And maybe we can convince you, some of you, to contribute back to the project. I think there's some really interesting things going on in here. Okay, so I actually, uh, in my biography on the website, it says I'm an admin for a small business. Well, actually, I just took a new job, so it's been a rush getting all my stuff together too. I just moved a few months ago, so I'm now in I work at the CSA, which you may— a few of you may be familiar with from our involvement in the Bro project, the Bro network monitoring platform. We work on that, so that's pretty cool. And a few other things about me: I do all the documentation mainly for the Nessus NG Toolkit. Okay, so let's get started with the topic. So it started in 2009 and It's basically done out of the free time of the two main developers. There's been some work where we've had a few other guys from industry where they report or help them do labor time to contribute back if they need something useful, like wireless RF monitoring and such things like that. So it's basically been people contributing in their free time, like most projects. And it's— even though it started in 2009, it's still in its infancy. But now we're at a good point, which we'll talk about later, to where we're developing, or it's at the point where there's a lot of features, very stable, and so on and so forth. So you can use the toolkit for network analysis, packet sniffing, traffic monitoring, security, all kinds of stuff like that, performance metrics. So the toolkit has a bunch of tools, and today you can quickly read over these if you need to. We'll talk about 4 of them. NetSniffNG, Traefgen, bpfc, and Flowtop. And those are the main ones, uh, particularly the— actually, NetSniffNG, Traefgen, and bpfc. But I will mention Flowtop briefly. So, uh, this is the big picture. So if you want to categorize this and what each of the tools do, um, this is how you can do it. This is a nice little graph that one of the guys created. So, uh, over in the traffic capturing and traffic generation areas, this is where we'll mainly be staying and talking about. So this is basically, we use in NessusMD, we use memory-mapped buffers on the PF packet sockets where the trap or the packets are memory-mapped from kernel space to user space, you don't have to do extra context switches or anything like that. So this is about the best performance you can get from user space. And there are third-party drivers like the Murakami Sniffer, Tenji, and you gotta be familiar with PFRE DNA. Those tend to outperform because they do custom things and they have specific hardware drivers for those cards to get extra performance. So one cool thing about Traefik, which we'll get into, is you get entire control over the packets, all the headers, all the data, and the toolkit is optimized for Linux. They're not interested really in porting, but if someone else will do it, they'll probably gladly accept it, but it's optimized for Linux. And it's a very small code footprint. So, and like I said, you're very good point about its lifecycle. The reason it's optimized for Linux is that the core developers of Netscape NG are kernel devs. That's what they do. So they usually have the latest features of the kernel, and you can see them on the kernel that's contributing back. And I'll go through a few of those. And if you're wanting to get it so we can use these examples, some of the latest features in the kernel, good idea is to actually just review the Netscape NG code because they usually integrate it when they find that it fits and it's there right away. An example of this is, uh, tpacketv3. So this is a new structure for packets and it was developed by Chetan Loek and, uh, it replaces tpacketv2 in the kernel. And basically it— the results, which is the important part for many of us, is just that there is a 15-20% reduction reported in CPU usage and a 20% increase in packet vector. And the only sniffer that has this that I'm aware of right now is the NetSniffNG toolkit, so NetSniffNG is the sniffing toolkit. And any program that uses libpcap is using TCPACKETD too, because they're using that library, that's what they have to compile over to use that version. So, um, there's a few reasons at the bottom, so packet density is increased, packets are closer together, and there's less misses in the TLB, so CPU can have more time to do processing without missing any cache. So another development that was recent, I believe this is from Daniel, one of our core developers in the kernel, is this neat little BPF instruction. This allows someone to— it's a dynamic instruction to actually write packet headers to disk when just truncating a payload, just skipping it, which is pretty cool. Because before you would have to do probably some math, you wanted to do it in the entire header, you know, the 25-byte IP header, blah blah blah, 8-byte UDP header, stuff like that. But now you can just enter this and dynamically does that. And basically at the bottom in the example, we have, you load this value, whatever the value of this POFF is put on the instruction, BTFSC, you return whatever is in the accumulator, and that would be the value, return packet, that basically specifies the size of the packet to return. So, which is pretty cool. And again, the Pcap does not have this. And ePFC is the only one where I've been at, unless you just, you know, write your own code to do it. And this is a new thing, this is also done by Daniel, where they actually have a net leak monitor. So you can actually, they have net leaks, so you can actually sniff on a PF packet socket and listen to net leak messages, which basically is how programs talk between kernel space and user space. So this is packet analyzer neutral, so we're actually going to have a, the plan is to have a dissector sometime in the future. So people can analyze those messages in NetSniffNG. Okay, and let's get to NetSniffNG in particular. So, it's an analyzer, recorder, replay player, and you'll see this on the website, the man page, it uses the PF packet socket and maprx and txring. At the time it came out, it was the first tool to use that, so you'll still see it using that. It's not too much of a big thing anymore, we've got libpcap and all those tools are reliable. They use— because of that, they use the RxRing. Anyways, so it is now— X-DefNG is the NetCon reporter for InSecurity Onion and Explico and the Network Security Toolkit distribution. So if you're using Security Onion, I'm sure there's a few users in there. Anybody got their hands? Raise them. Yes, awesome. I love Security Onion. So you're using X-DefNG in there as well. And we support— or X-DefNG supports a few different languages, which is nice because a lot of the tools you don't get the different IO web is. It's read/write, most tools use that, standard read/write calls. Here, snipng defaults to scatter/gather, which is a little faster, because it bundles up calls and uses vectors. And mmap is another faster than sequential I/O, or disk sequential I/O, I believe. And that's, if you can fit things in RAM, you should use mmap, because you'll get better performance. And we also, the Linux kernel has a JIT compiler, and it's not enabled by default, so when you run snipng, What will happen is it will enable the JIT compiler. It's just sitting in /proc/fs, or you can use sysctl if you just want to add it to the file. And basically what the JIT compiler does is that it creates a JIT image based on native code, or native— based on your architecture's code, so it's native. And then the result is that you compile that, and every time a packet comes in, that thing is executed. Against that code, and then because the code is based on your native underlying architecture, you get about— it's estimated about 50 nanoseconds extra performance over the normal BPF compiler. So you can use JIT to get a little better performance and execution with that. And it also does some basic OS tuning for a reception whenever you run sfng. It'll go through a few options in procfs like the rmem and default to max settings in the wmem, which basically just controls software memory buffers and increases the sizes. The Linux curl supports different packet types, so basically when you're calling the socket you can say, oh, here, apply only host traffic or broadcast traffic or multicast traffic. And NetSniffNG, you just add the --type and you get outgoing, others, multicast, broadcast, so you don't have to do a BPF filter really, apply these types and get the same result. Okay, so we've got a few examples here. Um, the first one shows a basic usage and how you want to do full content data capture. You say, hey, first I should note that the— it's really easy. So you just have in and out. It's pretty much the things, uh, you didn't know. It's pretty intuitive. So in on this interface, out to this directory. In that case, it says out to PCAPs. And we use silence because we don't want to print the packets to the screen because that will cause a number of interrupts because it's got to print to your terminal. And then we allocate a rate buffer size of 2 megabytes. And so in this example, we say, hey, we're going to run it as high priority. You can also run it as nice if you're running— uh, run it— rate it. And then we say we're going to run it at an interval of 24 hours. So every hour— every 24 hours, write out your pcap. So you have data to look at. So, got that built in. Here's a few other examples. We can use RFRAW, so that basically puts your device in monitor mode for using wireless. You can sniff raw. We can also replay from interface to interface, and at the bottom I'm going to show a few examples of line filters. I will mention that NIST Affinity does not completely rely on libpcap in the sense that it is compiled with libpcap, but we don't use the libpcap calls to open a socket and read packets and so forth. We actually just use it if someone wants to specify at the high level the pcap filter like tcp and port 22 like I have listed, it'll actually just go to the pcap file. So otherwise you can use bpfc. Okay, here's a little output at the bottom. I'm not sure how readable it is from up here, but generally we have at the top you use the verbose option, it gives you a few things like how many frames can fit in the buffer and so on based on the size of your buffer. And you can see the dissection in the middle, middle box, and it also has GeoIP built in, so at the bottom you can see, you may be able to see, it says that it detected a location in Sydney, Australia for the destination, which is kinda cool, so that's built in. And the statistics it gives you, which everybody's quite familiar with, incoming, passed filter, dropped, and then a percentage, and then capture duration. Okay, so let's go to TrapGen. TrapGen is a multi-threaded number generation tool. And basically you create a packet based on traffic language, a few expressions, and you basically just specify the hex values and you can generate the packet out the interface. Which is kind of— it's more flexible than that, but we get— there's a few macros and things that make things easier. And here's a test the developers did when they were doing his thesis on traffic generators, and you can see PacketGen is well known as one of the fastest ones if you don't have a hardware generator. It's a kernel module. And you can see that Traefik is fairly true with it, not quite there but very close. And has anybody heard of Nekojin? Okay, it's a pretty cool kernel module, it's in the source tree, so when you install the new Linux kernel you should have it installed. You can just load the module and then I'll give a few examples. But, so let's take a quick— dichotomize this and give a quick overview of each one. Packagin is typically faster on small packet sizes and nearly the same otherwise. Trapdin is not quite as fast on the smaller sizes, like 64 bytes and 128 bytes, but as it rises, as the size rises, it's much easier to generate Trapdin because it is hard to generate small packets at a fast rate. And here's an example on a 64-byte, one of my tests, I was generating 520,000 packets per second. And Packagen running the same test was 620,000, so 100,000 more. So, and the cool thing about Trapgen though, one thing it has as an advantage is that the entire packet is customizable, the payload and everything, and that is not the case with Packagen. You will, you have a few options and then you cannot really go any more granular. So we'll check that in a second. And then Packagen is configured by sending commands to ProcAdmin, so it's usually a shell script that floats around the internet. Values, the packaging commands, you just say set the, the command or the number of packages you want and all that, and CellScript does it, it basically echoes values to ProcEnvX, and you can find that on the web. Where in the case of TrapGen, what we do is we have a configuration file, and basically we just say hey, we make a couple heading delimiters, anything between these two points is going to be our packet, and then we just type it out, and then we generate it. And here's an example of using a 64-byte frame to package in it. And here's me not using the script, I'm just here to show you how it normally does. I just echo these values, and here you have source mag, destination mag, UDP ports, and then max port. So you can do ranges here, I don't want to do ranges, I want to generate a syslog 514. And that's basically how much, how much bandwidth there is for a packet. However, with TrapCam, you can do this, which is cool. Now, this takes a long time doing things like this, obviously, but with the macros, we can speed it up. Now, here's some performance metrics. This was at my house, it's at a server I was using lying around. It's probably 6 or 7 years old, so I know that it's reported from multiple people that we can do line rate, 1 gigabit line rate with TrapCam. I don't want to word any 10 gigabit per second performance metrics yet, But if I get my hands on some stuff that are on modern machines— I do have an example at the bottom, but like I said, my machine's 6 or 7 years old, still PowerEdge. Um, but here what I got, here's what I have. Uh, you can see if you look at the— let me get the rates— we have with the 64/120, you can see how the, the line rate percentage increases. So 64, we have 24% of the traffic generated was at that line rate. So, and we see that we're nearly right— excuse me— we are nearly line rate as we get to 256 and above. So we get results in the 90% range. However, in 10 gigabits per second, it's much, much harder. And I think on my test with the PacketGen for that 64, uh, packet went up 3% with TrapGen, but I was able to double that with PacketGen. I was able to get 6% of line rate with PacketGen. In 10 gigabit again, it's pretty hard to do that. OK, so let's look at a few traffic chain examples. So the first example we have, we're sending 50 million packets, reading in from this config file, it's called a 64.cfg, and we're outputting it to /dev/tube, and that's really all there is to it. So it's pretty simple to use, so once you have the configuration in place, you just hit enter and they're out, they're sent out the interface. And then we also did mortality tests, so basically if you want to fuzz a kernel or a network stack, you can do that. And you can also do with seed, there's an option for setting a seed. And then basically after it runs all the tests, it'll do like an icing gag out and see if the kernel responds still. So we can also generate raw material level parameters, which is nice. So you can do your raw, you know, if you do all that other stuff like beacon frames, you can just send those out, which I have a script I'll show at the, closer to the end, where I had written something like that. We can also pass through STDA. Where we say trapgen -e, that's a built-in TCP configuration. So basically what we do is we just pipe that into trapgen, and we specify on the next instance through the pipe that in, you put the dash, it's going to read that configuration from stdin, and then we'll output it. And here's a little look at the trapgen expression. So we have these macros, uh, fill, random, dynamic random, constants, number 1, incrementers, decrementers, And we have a few examples here. So, um, if you look at this under the macros, number 2, fill. So we can fill an example I have, um, add an 0xFF, so the full byte, all the bits set. We want to generate that 10 times, so 4 n bytes generate this. And you can— so if you want to generate 64-byte packet real quick off top, all Instead of doing a whole configuration, all you have to do is say fill , and then you get 64 bytes of hex. Not bad. So for different types that trapgen accepts, we have decimal, hexadecimal, octal, binary, and characters and strings. So you can specify things in other ways. You don't have to use hex. This is nice. Again, we can actually use the C preprocessor by passing the -cpp option and use definitions, and use C definitions. So we can include, uh, Crafty comes— when you install the SNMP Toolkit, there is a file stddef.h that has a bunch of definitions already, so you can just include that. And the example on the bottom, we have the protocol field in the Ethernet header, eth_p, described So if you don't know the IP above, you can just put in that definition, and it will translate. So, here's how you— this is basically a packet definition up here. You have the curly braces, beginning and end, and the packet data goes in between. Not too bad. So here is that example I was talking about earlier, the 64-byte packet of all Fs. I'm very nervous, so I have to stick in the file and tell Trapdoor to read it. And we have this one, this generates 512 bytes of random data. And you can use scripting languages. So if you guys want to make your own configs, here's an example of Python, uh, just kind of using Python print statement just to print the brackets, new line, and then add the data, and then multiply whatever happens at my packet times 8. So that string 8 times, and basically this, this, something that's on the top. And then all you have to do is send that with traefik in, and then here I use an sfnge to read it in. You can see that whenever it does the ASCII translation at the bottom with the characters, you see my packet, my packet, my packet. And the beginning part of it is actually in the source, and our source destination packages. So that's the other parts. That's why you see the cut off. Pretty neat. And here's a more, a little more advanced Traefik expression. So here we can actually tell Traefik, hey, you know, what CPUs do you want to use? Here would be 0 through 3, so 4 CPUs. Index 0. And we say Ethernet, we want to generate 12 bytes. So the source and destination Ethernet, max, it's hardware addresses. And then we say, hey, after that we want to use the IP protocol field, so we specify that in there. And then we want to generate an IP header of 20 bytes, random. I wouldn't do it, probably not going to be valid, but hey, we can do it. And then we'll do a— there are 8 bytes. And then for the payload data, we'll 14,172 bytes per enemy. So the random functions are the fun sectors. All right, so, uh, here's another example, a little more advanced. So basically what we're using is the built-in function here. See the top says trackn-e? Well, this is a built-in TCP example. It has a full-length packet that is valid. You can send this out, and it shows that we include the sdeff file, the header file, and then we fill the destination MAC, source MAC, protocol field, IP version, the internet header length, type of service, the total length of the header, and so on and so forth. And we actually got a macro for checksums, so you can just do CsumIP and specify the values based on the size, TCP ports, blah blah blah, and we can set the flags. So here's— we use bit shifting over here, a little more advanced, but all that can be done. Now if you want to use them in conjunction pack our pcap to Traefik. This is nice. You get a pcap file, I'm sure many of you are familiar with tcp replay, well you can do something similar with Traefik and sniffng, er, sniffng. So what we'll do is first we'll have the pcap, we'll write to disk, and then we'll pipe it out to Traefik. And you can apply a media filter in the bottom. I say hey, read in this pcap file, pack it up to pcap, send it out via stdout, and we want to take one packet of it and then just the TCP, we want to get the BPF for the first TCP packet in the packet clock, we pass it to trafgen and say, hey, generate that packet 1 million times, and then we do it. And we're hopefully going to have some more, we don't have, as far as TCP replay, where you get all the specific options with performance, with speeds and all that, we don't have that granularity of traffic, and that may be something to think about in the future. Uh, genconfig is a little shell script I wrote, and basically what it does is I have a bunch of packet configurations that I created, and I just stick them in there, and then I just pass some options to the command line, the arguments, and say, hey, I want my source address, source IP address to be this, my destination IP address to be this, my MAC address, blah blah blah, and port, and then it'll generate a custom configuration. I did the performance metrics, if you guys ever want to do that, I did the RFC 254 port, Ethernet size for instance, that's the benchmark methodology for Ethernet and a bunch of other network technologies. And basically, so if you just run that and you specify the IPs that you need to send to the machine and test it, it generates the entire set of 64, 128, 256, blah blah blah package sizes that are used for performance tests. So you can just do that. I have like GitHub link at the bottom, so you can check that out if you want to use it, if you want to contribute back, make your own configuration, improve the app so if it's useful, and just pass it back to Google. Include it, or you can't. All right, and I also have a, uh, one thing that's kind of cool is the beacon frame generator. So if you want to, you know, mass amounts of wireless beacon frames, you can do it. And I currently only have a 8-byte character limit because I didn't want to do the math, or I haven't done the math yet for calculating so the header comes out valid and all that. But so you can do your free Wi-Fi in that case. So you send that out, and it's back to the performance metrics. So how would we want If we want to do metrics, how can we— sorry. How would we proceed? Well, this is one way. We could do a— use the RFC 2544 generation set for a very different size. We can use Traefik. We can basically say, hey, let's generate a bunch of traffic to a machine that we want to send traffic to, we want to test the performance of. And then what we'll do is we'll run live-pps, which is again included in the Istio Finji Suite, but we'll talk about that. And that will actually measure the line rate. There's a nice option now, it's new, it's -P, capital P, where you can you can actually test the percentage of line rate coming in, so you can measure it that way. And we can then, if you want to do packet capturing performance tests, we can actually script, have, you know, an InstantNG run in a shell every 5 minutes or something, and have your Traefik instance transmit every 5 minutes, and then you have a script that says, hey, SIGINT every 5 or whatever, how long it's going to take to send the packets to the process of the sniffing, so you can piece together all those and do a test run. And then what you would do is you have that write to a pcap file that you can use capinfo, some auth later on, just to pipe out, you know, the loss metrics, all that. You can also pull the information from your driver and do performance tests that way. That's something I'm kind of working on. I haven't had a whole lot of time, but it will be done. So eventually I will have TCPdump, dumpcap, build daemon monitor, and sniff-ng tests and compare. So, you know, coming out in the future. Alright, so here's another example of the daily pcap logging. So here I put a lot more options, or instead of mentioning options, I'm just showing you an idea of a few of the things that I was working on to make sure I was ready for the presentation. So you can do prefixes, you can call it ALARM WARNING TEXT, begin with this word, my packet, other than .timestamp, well, .pcap or something for each one, and just synchronize the timestamp. So things like that can be done. And here we can also set SFP infinity, IRQ infinity, affinity, and so on. So we're doing high-performance packet capture. I think that's what a lot of people are interested in when they hear NetSniffNG, or that's why they go to it. And it's very hard to do. I've done a few tests, and it's a lot of system tuning. It's really, really difficult. You have to do a lot of tests on different buffer sizes, network card settings, kernel settings, socket settings, and so on and so forth. —dry to disk, like pdflush and all those dirty ratio settings. You gotta play with that. It's really hard to do, but here are some considerations if you're going to do some performance tests. One thing you should probably have is a network card that supports API, the new Application Programming Interface, which basically handles the interrupts better than they were handled before in kernel. Most modern cards have multiple queues, right? So We can use RSS, which is Receive Site Scaling, and essentially, say a NIC has 4 queues, what can happen is the packets can be split to different queues. And we can use RPS. RPS is Receive Packet Steering, and I have— there's a document in the Linux kernel source tree in the documentation directory called scaling_optics that explains all these. And if you don't have a card that supports RSS, you can use— RPS is neutral. And where you can assign queues. You can actually take the packets in and have them flow or balance kind of over your different CPUs. So you can set that up. And other things to consider are offloading measures. Usually offloading is a good thing to have. Performance, narrow throttling, coalescence. You should have multi-cores of CPU, lots of RAM, hardware locality settings. You should check out SAP Affinity. Fast disk configuration is very important, and so on and so forth. We won't spend too much time on that. And software configuration. So if you want the latest and the greatest, like all those new features I talked about early in the beginning, the Linux kernel, tpacketv3, the new BPF instructions, and so on and so forth, they are in the latest kernel. So you'll have to recompile the latest kernel and install it, and you'll be able to use those features. So when you're using it to do tests, and you're doing bleeding-edge stuff, then it's good to have the latest kernel, latest mainline for the latest fixes and features. So here's just a few options of ideas you can use to set or to tune your system. So here we increase the memory buffer with sysctl. We increase the queue length, that's the driver queue length by ifconfig. That's actually the qdisc length, I take that back. So we use the default qdisc FIFO, or fast, pfast, something like that. So you can set the value of that. Set IRFB affinity. Enable the JIT compiler yourself, just a big old -O 1 to the file and rock at that, and so on and so forth. And you can, if your disk is too slow, which is the case for many of us, you can use RAM, use a RAM disk that you just mount a 20— here, this is an example, mount a 20GB RAM disk, and then you can write to that, and then later on move it over to your hard drive when you just went at night or something, and then travel over there. And BPFC. So this is the Berkeley Packet Filter Compiler. I believe it was written to standard for the BPF paper that came out by Van Jacobson and, uh, his name escapes me, but, uh, I think I believe he follows it to a T. So we have support for internal Linux kernel extensions. So these are things that the Linux kernel supports that give you lower-level filtering access to the packets and will I'll show a few of these in the next slide, I believe it is. And along with libpcap, libpcap, uh, your Pi does not have those built in, so if you want to use those extensions at the moment, you'll need to use bpfc to keep it up to date on that. Also, we can use bpfc, we can use setcomp filters, where, this is something kind of new to me, I found this out not too long ago, where, uh, because Google Chrome uses this, where you can actually filter on system calls. So if you're using— people crawl on Linux Linux, of course, because this is a Linux thing. And so does SSHD, and I believe that's from the OpenSSH project, that SSHD. They use these as well. You can actually apply BPF filters and filter system calls, which is really cool. It's kind of like a sandbox. And then netfilter actually has— or iptables now, which you use to access netfilter— has a feature you can actually type in or read in a BPF files. We can generate a BPF compilation file or compile BPF code with bpfc and pass it to iptables, and it actually can output in the iptables for netfilter format. It's pretty specific. And here's an example. So we have a bpfc with a BPF file from R. We're basically compiling everything, the output, right into r.bpf, and that's all nested and iterated. So, uh, this is also a really good tool to learn how to write BPF, because you really can't do that with the Bcap, right? You just— it's quite a high-level filter in a compiler. Of course you can dump the assembler-like code, but here you can actually write it. So, uh, like I said, some of the kernel extensions that can't be expressed in the high-level syntax can be done in this, and there's a bunch of— a lot of other things that are instructions around here in the Bcap. So, um, And this is mainly used for kernel devs to do, you know, different, like, maybe there's bugs found, we'll use this to test this compiler versus another one. So here's some of the BPF extensions. So you can use packet length, you get the proto, which is the eth_type field, type, pofs, we're gonna talk about detecting the payload, interface index, VLANs, queues, CPUs, and hashes. There's those RX hash ones you can If you're using RSS, which is what's killing the different hashes to Valve and TrackMan, you can actually build your own hash. Um, so here's how you compile one. So, um, at the top we have a libpcap equivalent. So we say ether source, anyone reading this is gonna have source address. So a, b, c, c, d, e, f, f, p0, follow. And bpf-c is what I'm going to do. I just create a text file and I say load, LD. This is going to load 4 bytes, it is a word, at the 8-byte offset, which in the Ethernet, the frame, it begins with the destination MAC address first, then the source. So we want to look at the source, so we want to 8th in, we'll have 4 bytes left because of this Ethernet MAC address is 6 bytes. So basically that part will load the 4 bytes. As you see in the next instruction, it says just our jump if not equal So if it's not equal to the go to drop instruction, which is at the very end of the chain, very bottom, you see it's— if it jumps to that, then we're returning zero length of the packet, so don't return the packet at all. And then if it is equal to CCDEF, or MAC address, then we'll jump to the next instruction, which is load H. We're loading a half word there, 16 bytes— or excuse me, 16 bits, 2 bytes. And we're loading the extra— the other 2 bytes of space on the end of— so AABB, that's the rest of the frame, or the source MAC portion of the frame. And if that is equal, then we return 1514. If that is not AABB, then we'll return nothing. We don't match the MAC, and we don't return anything. And this is what happens after you compile, so you get the ugly outcomes. And you can use kcbounce, you can use -t to dump idle filters to outputs as well. And here's an example using the kernel feature from the extension. This is actually the filter on CPU. So we load the CPU value, and we say jump if equal to CPU— if it's equal to CPU 0, then we'll return. And negative 1 is just another way of saying return the entire thing no matter how long it's on. So we don't say 15, 14, whatever frame is even bigger than that, because the kernel will support it. Negative 1 returns the entire thing. Otherwise, if it's not CPU 0, then we'll return. So, and the example use case for this is if you're using RPS where packets are distributed across different CPUs, you can then use the filter like this to say, hey, I just want to grab the packets that arrived on this CPU when they're distributed. And then you run this, you say, hey, take it in and apply it, and it compiles in the kernel. And here's some notable examples where the NetSecMini toolkit was used. And two examples are— that I'm aware of— is that one of the kernel developers found a performance problem in the fragmentation memory in his kernel, and they were able to use Trapgen to create the packets to test that, and then later on they discovered there was a problem and they made a patch for it. And EPSC was used to prove and exploit a x86 JIT bug. So bug in the JIT compiler, someone used PPFC and found out that it didn't match, that whatever that was compiling in the JIT into the— in PPFC didn't match, and it turns out there was a bug in the JIT. So Ipps, or ifpps, is another cool tool. It just reads stats and reads it directly from the kernel statistics, all the proc files. So example, if you want to look at your network statistics, you can just do a watch on cat /proc/netdev, and it's got all your receiving packets, the bytes, the drop count, errors, the frames, all that. So you can just redirect it from there. And that's what ifbb— ifbbs does, along with a number of other places in the kernel, like sockirq, proc interrupts, and so on and so forth. And with this, since we don't rely on another tool, we're doing it directly from the kernel statistics, we don't have to use a library to make other calls, we actually get more accurate results under high speeds, high throughput. 'Cause an example was on one of the dev machines where they were generating line rates on 1 gigabit Ethernet. IP track, which uses libpcap, was only reporting 20.6 packets per second, whereas if they were reading directly from the kernel stats, ifbps reported the full line rate. And this is Jon Schipp. Basically, all this stuff will be fine if you use the normal system interface. VMstat, all that stuff's printed here, so you can monitor context switches, idle time, CPU time, idle wait time, all that stuff. So when you're using network traffic performance, or you measure that, it's a really nice chart, and you can output it to any platform. So here's a look at ip ps. This is not looking fancy right now because this machine just has a single CPU, and as you increase the hardware, the display gets bigger, you see a lot more stuff. So that is generally IPPS. This runs in a loop, and the default is 1 second. And we can actually output CSV, which— anyways, not CSV, that's what they call it. And basically, this is a space, a whitespace-delimited Hadoop plot. And here's an example where I'm reading an IPPS CSV file, and I'm just saying, hey, print all the lines and max packets per second, and which is the third field. And here you can use Gawk because it has a timestamp function. Where we can say, hey, every time you convert it, it's like call it CDC file. And here we say, hey, print every line or interval that matches or where the drop count is greater than 0 and the errors are greater than 0. So we want to know if we drop packets for that, for our testing interval. And you can see here at the time we converted it. And so you can hack it up easily with MOP if we need to. And here's a quick GNU plot chart of a test I did And I see that you want to play around that code at the top. This is— I've just begun to use like new plots. These are really rudimentary, but, um, essentially we're just getting a run and we are— you see the, the chain or the top green line as it goes over, that's where the trapped in, uh, RAMs are. And, um, from another machine, and this is on the capture machine, this graph is generated from that, those stats. And we got about 650,000 packets per second, and then we had a loss of like 50,000 or so drops from the kernel. So you can graph that and you can go a little further with the same file and everything and just say, hey, look at the drop count, and you can see that it's a little less than 40,000. So you can show that you put the time on there and correlate with my ipfs stats to see, you know, the CPU time slightly with the cron job in the background. CPU time or something of that type. So you can just pull in all that information if it's written. And then the other tool I want to talk about is Flowtop. So Flowtop is a top-of-line NetFlow viewer, or connection viewer, user of Netfilter. So basically you would say an iptables command, you would say, hey, pass in all this traffic, and you run Flowtop, and just anything that comes in that passes that match rule, it links, or that's how Flowtop basically works. Call it Netfilter, and basically it actually gives you a nice little flow. This could be good for instant response, by the way, because it actually tells you what program on the machine that you're running it, uh, you have to generate the traffic versus receiving the traffic, which is really cool. So I appreciate that. And eventually there's going to be some new features on here with sorting on bytes, packets, and all that. So, um, you can also notice the GeoIP location. As you can see, we have Sacramento, California, and Mountain View, and so on. Much greater. And to install this, if you're running Ubuntu, this is generally like, so let's put this slide up. Not too bad. And what could be next? Well, um, we have a few things that are going to be worked on. One thing I'm kind of excited about when it comes out is the packet fanout, where you can fanout packets to multiple sockets, have multiple listeners, and distribute the load packets across multiple instances. So something to look for that's fairly new in this firmware too, so I don't know where the state is. And then, um, basically pretty much everything else I mentioned with the flowtop, we'll apply some packet counters, and there's going to be, uh, Malsatoll. That program is a nice HP-like program where you can generate packets at a higher level. That you can generate Cisco packets and stuff. And it's got a bunch of dissectors built in to where you say, hey, I'm gonna generate a Cisco packet, I specify the facility, the severity, blah blah blah, it just synthesizes it out. So it's really cool stuff like that. And that is a tool that's been picked up, the man that developed it originally passed away 2 years ago, and so they're trying to integrate that into the toolkit now, keeping up his code. So it still needs to be improved. And, um, there's a bunch of links if you guys have any more thorough research information, go to those. And if you want to look for flight configurations online, not a whole lot there yet, but if you use those Google dorks at the bottom I put on there, you can probably find some filters or some traffic box. All right, do I have any questions? No questions? All right, I'll help you guys check out the toolkit. If you have any questions, do email me or email the list. Preferably the list, unless it's more personal to my talk. And I guess that'd be about all. Thank you guys.