LUG Meeting (Linux Kernel Capabilities) 02-23-2015

Alright, log meeting here. Some group updates. We have— our GitHub account has been transferred or turned into a GitHub organization. So if anybody in this room would like to be, or anybody in remote or wherever would like to be part of the organization so they can contribute to our repositories, do let us know. The organization gives us more collaborative control. We can put people on a team, we can assign teams to specific projects. So that helps a lot. And the other big thing is that in the past, here's where I wanted to go make a repository. So I would have to log into the personal account for using ECM log and the password you had for it. But, and then switch back to my account to use my screen. But now, this organization, you create the repository as your user and supply it or assign it to the organization so it's still underneath the organization. Our Zen project system is back up and running. So we pulled out, there's two bad memory sticks, we pulled out both DIMMs and it's now back up. What's nice, so now we, the GitLab server is accessible, and it goes to the next bullet point here is that we do have a GitLab server. So if you go to git.gnulug.org, and we have our own personal system where you can actually have your own repositories and make private repositories as you want. So we're going to be hosting a lot of our stuff from the two groups, OpenSM and LUG, that, uh, we want things to keep confidential, like maybe system configurations and stuff, and we'll put them here and make them private. So if you would like an account, let us know. That way you don't have to pay for GitHub or whoever else. You can host it with us. Links in the news. Not a whole lot here. Ubuntu— there's probably plenty, but it's all I came up with. Ubuntu 14.04.2 released. You can get the ISO. Has a new XStack and a new kernel for 2-week updates work with it. GDB 7.9. Was released for those interested in debugging. I think it now has more of a better scripting language as well with that release. We're going into education epoch where we talk about education resources. So the Vim-L Primer book just came out a month ago or so, and it's a book on Vim scripting. It's actually pretty short, like 92 pages or something, but the author of it I got in contact with He gave me a free copy. My goal, or what I had to do, was just I had to review it whenever I finished the book, but it looks like he may actually give a copy to everybody in luck. So, oh, free ebook, ebook. So, um, if anybody's interested in that, let me know. Um, Lucian, I'm hoping you'd be interested in this. Yeah, I figured you would. So, uh, we'll see how they go until they get back with me. She can talk to the publisher and see if we'll do that. Um, moving into the talk period. Ptrace, we did a Ptrace talk last time and we're talking specific kind of process file called ptrscope and we wanna zoom down here. Actually, I don't even remember what this did. I already forgot. Take a look here. Okay, so it controls whether you can actually ptrace a child process or not based on your user level. So ptrace directly from a parent to a child. You can use— or strace -f. There's the root user. Let's see here. Ah, okay. Sorry. So what this actually does is— so ptrace or ptrace— So there's a system call called prcontrol that a programmer's gonna apply to the program and say, hey, you are either allowed or not allowed to be traced by application. And ptrace thus means the use of strace. Strace uses ptrace in the kernel to actually log on entry and exit of system calls so it can record it. And what actually this option does instead of actually having to be able to do that in the programming language, you can control the behavior by this this variable here or this profile, the ProcAttach. And it says, so basically if you want to attach to a process that's already running, this is, if it's set to 1, you will not be allowed to do that. But so for example, sshd fork. So if you attach to strace sshd and then just the f option to actually trace all the fork processes underneath, all the child,, then you'd be able to do that from the command line. But you can't do that if one is set from the— if the process is actually running because the ptrace scope only works if you actually launch the parent and then you're tracing the child. So if the parent's already running and the child already has child, you cannot walk into that and trace that. So that's the big thing with the ptrace scope variable controls. Moving on. Nothing for xspace protection. Shell space. So I did a 20%— I have on this maldns search script I wrote a few years ago. I did an update to it earlier this week. Actually, it was the weekend. And I did a 20% performance increase by switching the use of the Bash built-in, the brackets for testing, right, with what is a Bash keyword, the double brackets, which is newer. And the difference is, besides that there's some expression pattern matching differences, the big difference for the performance increase though was the fact that the double brackets does not spawn a new process every time it's called, but the single brackets do. So that's what they call the built-in that actually spawns a new process every time a test is made. So let's actually go into the code here. You can see it's real simple. So I have the single brackets. All I did was change them to double brackets. So I just added a bracket to each side. And that caused by itself a 20% speedup in my program. Simply because every time you called the single bracket, it had to spawn a new process. In this case, it doesn't. And then now Justin Azoll for the Rogue Team, uh, did a patch as well. And now there's a 100% performance increase because we're now using associative arrays. So instead of doing a linear search, where I'm reading it through, matching all that, reading the next entry, matching that, reading the next entry to see if it matches any other piece of text. This is just IP list and stuff. So here's the IP address in this list. I need to match against my, my, my raw log or something and then read through this next one before I go through it. So it actually had to do that. Won't go through it multiple times. You just do it once but it has to be stored all in an array and you do one single pass through it. That was 100% Important to note though is that if you use the double bracket, you lose POSIX compatibility. That's true, yes. So make sure you call it /bin/bash and not /bin/sh. Very good, thank you. I guess we can go ahead and demonstrate the use of that so you guys can see what I'm talking about. So let's do— Okay, so for i in— let's go ahead and expand this to 1,000. That might not be enough, but do— we're going to test for i here. Done. Time. Let's do— Does that spawn a new process for every single one of those tests? In this particular case, we're just testing that i is— that there is a value for i, that it's there. You can see it took 10 seconds. Let's do this way now. 5 seconds. So half the time in this particular case. Right? Simply because we don't have to spawn a new one every single time. And also note on UNIX systems, the older UNIX, the history behind this, you see a program called test, right? That's in the old UNIX systems. I guess System V, I think, or VII, they came out with. So you'd have to do, you test your variable test and you do, you know, like bash for something, right? You test it that way. And if it exists, it's 0. If it doesn't, then it's a It doesn't. So let's try one that does exist. I'm sorry, it doesn't exist. And I see it's a 1 now because that variable doesn't exist. But we're actually doing the same thing when we're doing this. So in this particular case, we're going to do home again. The home variable says— I'm just going to go completely through it. Home is set to this, right? But if I want to test it, it prints it. It exits with a return code of 0. If that variable is set to a value, and you can see this here, it is. But if it's not set to a value because we made it up, made up variable, see there's nothing there. And we do test again just one more time, and then we do made up, and then we check the exit code, and it's 1 because there's nothing there, nothing set. But at the same time, you can do this, made up, and that returns 1 as well. But if it was home, The value is set to my home directory, so that returns 0 because it exists. The value is set. So you can actually see that there is a program for compatibility. They actually, if your shell didn't support it, they actually have a program called the bracket. It's literally the binary of the bracket. So let's do which. Alright, there it is. So look at that. /bin/the_bracket. That's a real program. So do, whatever I'm doing is from Bash, type type. Actually, I don't have a— yep, yep, yep. Okay, so the shell is built in. You can see I'm actually not calling the program, but if I wanted to actually call that particular program, I could do it as well and do the test that way. So just a little bit of information on that. Moving on to Container City. So this article came out recently and it was really interesting. A guy is using Docker to run all of his applications from the desktop inside these Docker containers. And it's kind of cool. So he's just doing it for fun. So he's running this ERC client inside Docker container. And actually, please— you can see there's what he's running. So he's bind mounting his ERC configuration from the actual host inside the container. He's setting it read-only because you don't actually need to write anything to disk in the file system there. Another one is using MUT. Here he's bind mounting his GPG key from the host, etc. Also setting the tire, the local tire. I don't know what the hell Ravenwood Stream is. Oh yeah, it is. It's like, okay, clear client. So he's bind mounting that, the top disk configuration for that. I'll check that out. Same with links, but he goes further. He does GUIs with X11 forwarding. So in this case, he's running Chrome in a container just on his desktop. And you can see that he's actually bind mounting the X11 Unix socket on the machine so he can communicate with the host, right? So he can display the variable and then just running Chrome. And then here's the result. He's talking to this girl in a container that's presented on his host. So that's pretty cool. And if you want to actually get sound from the container to your host, you have to bind mount the dev sound device. This is a pretty cool product. You get Spotify as well and a few other things. What is this one? Departed, a partitioning program, Skype. But those are some really cool products. And Tor Browser in a container. So that's kind of nice. All right. So the next item in the container cities, the addition and dropping of capabilities in the latest kernel for containers. We'll talk about that towards the end because it's going to go right into my talk. Devon's here— not here tonight, so we're going to speak harder. Does anyone else have anything? I have a short one. Sweet. Yeah. Ready for now? Yeah. Alright. I was going to do BitFanging with libfdi. Okay, sounds good. But I have to use my own hardware. Okay, gotcha. So if there's anyone remotely attending, let's give 5 minutes of hold. Okay. So, um, there's something for that. We need— Yeah, yeah, that's right. I can't wait to, uh, Zoom's— I can't wait until Zoom supports Linux. That was pretty good. So this is one— I get tired of pressing the Escape key. I haven't really taught myself memory motion for this yet, but, uh, or motor memory. But you can actually, instead of having to press Escape to exit normal mode, you can switch it with Ctrl+left bracket. So I'm typing something in search mode, right? Oh wait, I'm not even plugged in. Whoops. So you can use this right here to actually— good old— Alright, so I'm in insert mode right here. You can see insert at the bottom left. So I'm gonna now press that and I can now do stuff, right? So it just switches it out for me. Fairly simple. Insert this and then I can just normal commands, right? And this, I would say this applies to any terminal program because the way the terminal escape sequences work, it probably knows you are pressing escape. It's just, that's the sequence that it sends. So that will work with anything in the terminal. Max, do you have an example? I can try to think of one. Let's see. I think in Bash, if you either hit Escape or hold down Escape, it asks you if you want to show all the completions or something. Oh, I've done that a long time ago. It's not doing that for me right now. Because you know how when you hit Tab, like Tab Tab, it does that too. Right. Yeah, pretty sure escape does that sometimes. I know, what is it, set -o vi is a vi input mode. Right. Unless you have a specific— Yeah, you have to get a point. —version-based program or something. Right, you have to get a point setting it to vi. Mine's default Emacs. Yeah. Let me try to do this in here. Well, it's just the terminal thing, so. But yeah, I didn't know if it was OS X the version, because I'm running an old version of Bash. I don't know. Okay. Yeah, so that's good to know. Cool. And then the other one we have is show compiled options in Vim. So type version at the bottom, right? It tells you all the options that Vim is compiled with. So if you need a specific thing, you can match it here. And you may have to recompile it or get a new package that has a specific feature if it doesn't have what you need. One— go ahead. Sorry. It's equivalent to the long-form option --version too. So if you don't want to open it, if you just want to call it from a script or something, you can do --version and it's the same output. Oh, nice. Thanks for letting me know about that one. Cool. And the one that I wanted to mention that I hadn't— there's one for pasting from your clipboard to Vim. And OS X doesn't have support for it. Wouldn't that be really nice to have instead of having to copy and paste it and then do my— And they never have that compiled in in any of the binaries I've ever seen. Yeah, yeah, they don't. But that's a nifty one. And then same from Vim to the host system clipboard too. There's a register for that. Oh yeah. Yeah, you can paste it back in. But I think for OS X now you can use GVim. I think it works that way. Anyway, that's a little thing. The other stuff I was going to talk about, I talked about strace last week and I actually had an example of a real use case like a few days later at work of using it to solve a problem. I don't have an actual session I can replicate it but essentially what happens is, so I was working on a program called Segen. It's a multi-threaded log analysis. Engine and it required some dependencies. One of the dependencies was liblognorm and I created the package out of it and then I transferred to another Linux system and installed the package after I built it and all that. And so it worked fine on this one system but not on the other one. And the point is whenever I ran Sagan, it would fail out. The runtime linker would complain and say, you know, library not found. Okay, so what I had to do, well, I had to figure out why the hell wasn't it there because it was the location, it was in the same location, the operating systems were the same. But so whenever I ran the program, the runtime linker would go through all the libraries and look for a specific place. But I didn't actually know where it was looking for the library, right, to load it. And it turned out that it was in a location that's not enabled by default or set by default in the runtime linker. I actually had to manually put that in the LDSoCache and recompile that database and then it worked fine. I guess I can kind of show them how I'm gonna— that probably sounds like a bunch of junk. So anytime you open a program like— so we're using strace, we're using -e this time, option I didn't talk about last time. It actually is only gonna print the system calls for that system call that you specify -e. So when a program runs like ls, it's actually gonna open up a bunch of libraries it needs to actually get the functions it requires to run, right? So by default, this is what it runs. It looks in the ld.so cache. This is the runtime linker cache in Linux. This is the cache of where everything else— where all libraries should be found. And you can see it actually runs nssylinks.so and it requires the eclmc, the ptr-a, and a few other ones. And then it opens /proc filesystem and then it opens a file for locale information. So that whenever you run it, it can print it based on your locality. So in this case, actually I might be able to do something. I'm just thought— so /usr/local. So we want to move— I don't want to break anything. I'll break this up. So libpcr— should I move that? Let's see. Move libpcr— wait, hold on. Usr/local. Okay, so let's move libx86/libpcre and we'll move that to usr/local/lib. Alright, so ls. Oh, I gotta rebuild the cache. I think that's why it still works. So we're gonna ldconfig. When you run that command, it actually goes through all the paths to the libraries and builds the cache file for you. -v will actually show you each library it's building and putting in the cache file and their location. So here's userlib, there's just straight up lib, and the other ones. So that one we were using was— let me say, new. So let's actually just search. Just go ahead and ldconfig -v and then search for libpcre. Hopefully it won't be there. Goddammit. What? You moved it to a different one. Oh. Thank you. I didn't— thank you, thank you, I didn't even notice that. Um, so let's go ahead and do the reverse of this. I'm feeling so hard tonight. Why is it not coming up? I don't know. Anyways, okay, so I don't know where that is. I don't know what the hell that is, but we'll just start from scratch. So move /usr/lib/x86, then we want to do libpcr. Thanks for catching that. There's a few different ones. So why— oops. We want to do ls PCR. Oh, it wasn't PCR. I think it's right. All right. So there's the library located there. Oh, it needs a symlink too. So it's a symlink to 3.1. 13.1. And what we're going to do is we're going to move this library, 13, to /usr/local/lib. Okay, now ls should fail. Yes, exactly what I wanted. So at this point in time, this is actually the error I had for a different program, right? It said it couldn't find liblogmore. So the runtime linker unable to find it. Ah, damn it. I rebuilt the cache and I guess it found it because it's in Ubuntu's cache. I shouldn't have done that. But you saw the error message and what we actually wanted to do is we want to run strace which is how I figured out where it was looking for the library. So first before I do that, Just go to SO. This is where the runtime linker, those configuration files here that tell it where to look for files. So let's just do an fgrep /usr/local/lib. Okay, so lib.c has that. So that's why it rebuilt the cache whenever I moved it. So let's actually comment that out. So this is important. So if you're ever actually working with libraries and the system doesn't know how to find them, this is exactly what you do. You just go in the directory in the runtime linker configuration directory and you make sure the path is in one of these. And it tells you where to search for those particular libraries. Then when you're done editing it, you run ldconfig to rebuild the cache of the libraries. Now ls should fail and it did. So let's just take a look. Here's why it's debugged. So I didn't actually know ahead of time that you needed that in the runtime linker path. So what I did was open ls and you can see now, look at our output. It's a lot different. It's Found these three, but everything after says E-N-O-E-N-T, no such file or directory. So it actually, this is what's actually happening, it's trying to run, it's trying to find libpcre for every path that's in those configuration files, one after another, and it doesn't exist. Eventually it tries all things and it's not gonna work because again, the library's not gonna be able to be loaded if it doesn't exist. So all I had to do was working with That program, libmsg, actually liblogarm, once I saw that, I go, oh, it's not even looking in the right directory. It's like, I installed the /usr/local/lib, but the strace output shows actually not trying that directory at all. So that's why I didn't find it. Oh, I just need to add it to the runtime. Fun, fun. Then these PCRE also use that. I'll just do something a little less Sorry, it's been a long time since I used that. So /usr/local/lib and then we'll do— that's actually pretty funny. All right. Yeah, we're back working now. Nice. I don't even know if I could have done that with edit anymore. Let's try it. R is for file. L, V, F, S, O, C. Okay, so print. And then E for edit? No? How do you edit it? I can't remember. Is it write? Write saves the file. I know there's I and A for insert and append. Ah, yeah, okay, that makes sense. Right, and then let me quit. There we go. Let's see if it works. Yeah, because for the— you have to do like I, Enter, and then you type what you want, then you hit period, and then that exits out. Oh, okay. Yeah, I don't remember how to do this. I had to use it in 1.8, I think I told you about. I haven't used it since. I just messed up. Ridiculous. That's funny. Anyway, that's runtime linker debugging right there. Probably explained it pretty poorly because I kept screwing up. So, the slide's big talk is on Linux kernel capabilities. So, this is something I've used in the past but never really thoroughly, fully understood until I started— I shouldn't say fully now, but I have a much better understanding. So, I was like, hey, I'm going to learn something new. A lot of people don't know about this. So, this is more of a 14th slide plug essentially. So, Capabilities are privilege set— or they have capabilities have privilege sets in the Linux kernel. So capabilities were designed to divide up the power of root into fine-grained controls. Instead of giving you control— set user ID on a binary and having the ownership of the binary as root so that anybody that runs it, they become— they affect that user ID as root. Instead of giving them root powers, what you do is you divide up root and apply these fine-grained controls to your particular programs. And there's two ways that this has been done. Earlier on, in earlier kernels, it was through threads. You actually had to make a system call to apply the capability to a process. But as of kernel 2.6.24, the apply— the program that actually gives filesystem capabilities. So what actually happens when you apply the capability to the program, you run the capability program and the binary like /usr/bin/ls and you apply the capability to that program. What it actually does, it actually writes capability information to the inode structure. So you can do that with a tool that you can use to view the inodes. And that's how it's different. And the program doesn't actually need to set the capability. You don't have to be a programmer to want to edit your program just so you can run it at a certain privileges, right? You can actually just apply it to the file system in the kernel. Whenever the program is run, the kernel, it will ask the kernel or the kernel will actually check for what the capabilities are and then choose which one, see if they're allowed or not and then run it. So for security, this is very important in Linux. The other thing is, things that a root user bypasses all capabilities, the checks. So if you're a root user, it doesn't really matter. But if you are a non-privileged user, anything that is not user ID 0, then you're actually, the kernel, once you run the program, will actually check for those capabilities. So the manual page for capabilities is man 7, or man 7 capabilities. It's the manual section 7. And you can see there's a bunch of them. Each one is a specific kernel capability. And you can see here's one that you'll use if you work with packet sniffers or programs that need, require root access to actually maybe sniff raw sockets on a network interface or open a network interface and set it in promiscuous mode. You actually need to have capnetraw set or you need to use sudo or you need to become root. Otherwise you're unable to. But instead of giving someone sudo access and instead of making them root, we can divide the security up into these specific granular controls again. And so we can actually apply capnetraw to a binary and then when you run that binary it actually asks the kernel if you have permission for that see what it does, and then it will run it so you can actually just use the privileges for that program that you set. Now there's 36 Linux kernel capabilities at this point in time. There are a few tools that you can use to actually get the kernel capabilities and set them. If we're talking about file system capabilities, remember there's also thread-based capabilities. If you're a programmer, you have to call appropriate system calls. So first of all, the package on Linux for capabilities is called libcap. In Ubuntu, we can actually use apt-filelist to actually see the contents of this. They actually have cap2, so I guess version 2 probably or something. That's what they call their package. Okay, so I'm viewing the contents of this particular package. You can see that you have the library, right? And then you have some programs that are in /sbin. These are the programs that you actually use to check the capabilities set on your binaries. And capabilities only work on binaries, the particular file. Since it works with inode, it uses inode structure to actually contain the capabilities for it. You cannot use symlinks, for example, because that's a separate inode for the symlink, right? So what we're actually doing is we wanna use getcap to get the capabilities of a program. We use setcap to actually set them. And getpcaps is another one, but this one actually gets the capabilities of a running process. So we can actually check that. So I wrote this little for loop and actually we'll get the running process and the printer capabilities. So you'll see that we're using pgrep -f and we're just matching all processes. So you can see that this is what this does. F just prints the process ID. So it's printing all the processes running. Simpler than using ps aux and then having to parse out the first column or something. I guess you could do the -o PID too, but that's This is how I did it. So at this point we're gonna run through and for every process ID we're gonna look at with GpCap. And you can see at this point here's the capabilities for each one. So some of these like this particular program, whatever the hell it is, has capabilities for binding, for network administration, for network raw, for a module. See that's 418. So let's just look up that program, see what process that is. 418. And that's dhclient. So it needs all those things for hash lookups on the interface, etc., for requests, etc., and responses. So now we can see that even default Linux, how important capabilities are, that there's a lot of programs that use them by default so they don't have to get the user root to be able to run them. So, um, let's say check them. Another way we can do this, we use getcap to actually do it for file system capabilities, check it on the file system. So it's not clear at this point whether these capabilities are set through the program via syscalls or whether these capabilities are set in the file system by the inode structure. So in this particular case, we're gonna do getpcap and we're gonna do which or getcap and which and we do dhclient and— why didn't that— oh, that's because that's set through the system call, I believe. That's why that didn't work. Interesting. Let's try another one. Who would have get— or get cap— let's do /bin/ping. Nope. Let's set one though. So we'll go ahead and do set cap. We'll do cap net raw ping doesn't need this. The ep, I'll explain that in a second. /bin/ping. At this point we'll do get ping. Now you can see that that one actually has a file system. Attributes, right? So, well, there's a lot of instances that git-calf on— I'm just making this assumption, I haven't actually read this or anything, but what it seems to me is that if you use the git-calf and it's not gonna pull from the file system attributes inode structure, then pretty much I think to me implies that it was set through the system call of that particular program. And actually, you know what, we can test that with strace. 'Cause strace is the shit. So what is the system call for capabilities you can have. Caps set. Get set capabilities of threads. So let's actually see if that does it. So what we're going to do is do strace -e cap_set and dhclient. Oh, it's got a— it's going to— it's working. The client and the challenge are exiting, so hold on. Ah, just tying it off real quick. So dhclient -r, I think. That's renew. Son of a bitch. I meant to release it, I think. We're just gonna— well, my server system stopped responding. Ah, there you are, SSH. Let's see if I can make that on screen. Turn her off. No! I wonder why that happened. Oh, I know. I'm an idiot. They have to actually match that up as they say. They have to save it to the box when I just— dammit. Alright, so failures. Alright, so what we're going to do is— yeah, I'm definitely feeling hard tonight. But that's okay. We're learning as we go. We'll get back to that in a second. So while that's reloading— so there's two types of capabilities I talked about. There's file system capabilities and there's thread-based capabilities. Each capability Those types are called supersets. And each of those have 3 subsets underneath them. And they're called permitted, effective, and inherited. And they actually have different meanings for each set, each superset. It's kind of complicated and I hate it, but it's how it is. We don't actually need to understand the differences. I'll show you the man page where you can read about them. But essentially in the thread, I remember the thread one was fairly easy. Permitted and effective, the difference between the two is permitted actually has the list of— it's a bitmask of capabilities that are allowed, the program is allowed to use. The effective set is the capabilities that when you run the program, they're going to be in effect. They don't actually— so essentially the permitted has to be there and then it has to be set in effective. But the reason you have two different ones is because you can actually not— you can actually turn it off with the system call so that it's still in permitted but you're not actually utilizing it. So it's a way to drop and add these capabilities. Capabilities in your program. So that's why we have two separate sets, permitted and effective. So effective is when they're in effect. Permitted says this is your baseline, you can only use these available. Okay. So that's kind of interesting. And then capabilities are disabled if LD_LIBRARY_PATH is set. So that is a variable. In your shell that tells you where to look for additional libraries, right? And the reason this is a security mechanism, if you can set all the library_path, you could then call your own libraries for things and not use the system libraries. So you can do malicious things. So if a program requires capabilities and that says no, this is no good, we're probably not gonna attempt to do it anyway because this could be, an attacker could be using a malicious library. Or something, calling their own functions and calls. So that's why it does not work as soon as library— load library is set or path is set, then it is disabled. Essentially, that's pretty much capabilities, but now I want to— we got to talk about actually practical uses in just a moment. So can I get back in now? Yes, okay. So let's find a program that's not— that's not Qt Client. All right, so what is— So I was probably hit to— let's do something that's— oh, that's pretty much— you know what 1411 is? I'm guessing that's these ones too. 1411? Yep. So I saw another one, 721. 721. And there's RPC. Yeah, I don't need that. OK, cool. So we can actually use this one. All right, so what we're going to do is we're going to bat v. I don't even know what that is. Okay, stop it. And then what we're going to do is— so statd, so we're going to do which statd, make sure that we're actually using process capabilities or thread capabilities. Why did that not work? I'll look at step E. Oh, didn't resolve. So where the hell is step E? It's not called that. So in the script, let's see, it's that these steps and— oh, it's format. So format command, right? I don't know where the hell is that here. Jesus H. Christ, what is this all? Oh, there's more to read. Um, SM-35, is that right? Exec ARP, oh, ARP, there it is. Durr g'durr. All right, so Yes. Alright, so we want to do git cat, right, and we want to do /sbin/rpc.d and nothing, right? So now we're gonna strace it and what we're gonna do is use, I don't remember what that was, capset, alright? Jon Schipp. Yes, capset. So we're gonna use strace, look for the capset system call only. And then /sbin/rpc.statd. And what did that exit? Oh, maybe I'm— oh wait, it's 4-bit, right? That's what's going on. pkill rpc.statd. It's kind of raining. Alright, so let's actually do it again and it will work because I'm expecting this to work if it's Capsets and RPC.statd and again I need to add the -f option so that it looks at the analysis ports. Ah, yes, perfect. Okay, so here we actually prove that's how it's getting its capabilities, right? It's calling the syscall capset. Sorry, that took so damn long. And then it's passing all these parameters to the system call. As you can see, it's setting each one. Which is where you'll get it. So now let's actually use some— or talk about something. So tcpdump, perfect example. You want to allow your user to run tcpdump on your system, right? You don't want to give them root though because to run tcpdump in promiscuous mode requires a bind to a pf packet socket. So that requires root. So in this particular case, I'm root, I can run tcpdump, great. I'm not root, I can't run tcpdump. Well, I just need a special option in this case. And I don't have permission to capture on that device, right? So how can I fix this? How can I allow my user just to do TCPdump, have privileges, special privileges for that and no other program? Well, I can use sudo or I can use capabilities. Let's take advantage of capabilities because they're more plain grade. And in this particular case, what we're gonna do is we're gonna set capability cap_net_raw +ep. In this case, these are required. You have to So the E, so this is the EPI, those 3 capabilities that I talked about. E is for effective, P is for permitted, I is for inherited. We only need effective and permitted in this particular case. So now what we're gonna do is we're gonna apply it to the TCPdump binary. Should we, nope. /usr/sbin/tcpdump? Yeah, okay. So we're gonna apply it to the binary. Okay, now what we're gonna do, cat /usr/sbin/tcpdump. And verify results. You can see that the capability is now set. Now what we're going to do is we're going to drop off, become the standard user who could not use TCPdump before and try it now. And now that user can use TCPdump, right? So it's— yeah, curl capabilities are pretty sweet. And I can tell you, using— and this is good to know too, Waylon, for ILLID in Docker is that when you're doing capabilities, like for BroCon, we did sudo at first. It was a pain. It was just kind of, it's kind of get up, give them, give them access to go to the Bro binary. We limit to the binary, but that means they can use the exec function to actually run the program stuff as root. So we actually eventually ran capabilities inside Docker. So you tell Docker, hey, I want to add the capability to the container. And by default, Docker drops capabilities. It only has a set number that are actually enabled and drops the rest of the security concerns. And what you can do is you have to tell Docker, well, I want to enable netcap raw for this. We use Bro, for example. So BroCon, for our particular image, we had to set netcap raw on the Bro binary so the users could not become root in the container, but they could just run Bro as a local standard user. And let's actually just go quickly to the example of that. Dockerfiles here. And we're gonna go to Bro and Dockerfile. And right here, this last line of the Dockerfile, setcap, we're actually giving Bro setnet raw and setnet admin, and we're sending it to the Bro binary, right? So that way the users don't have to become root in the container. But this, by default, if you just do this, it will not work because remember Docker drops the privileges. And if you try to run that program after, it will fail. This goes back to the Docker portion of the early Container City meeting section. You have to specify prior to running the container --cap-add, --cap-drop for which capabilities you need, and then once you're in the container, you set the binaries. So With this, while I was doing this research for this talk, I was like, wouldn't it be really cool to add an easy way for IOLITE to support all these capabilities? So IOLITE is that training system software I wrote. IOLITE now has support for all 36 Linux kernel capabilities in a yes or no format. So let's go to commits. I did this 2 days ago. Config file toward the bottom, I think. Yeah, here it is. So at this point in ILLID, that's one of the new features in ILLID. So now there's a new configuration file in the /etc/yllid directory called security.conf, and you can just set your kernel capabilities right here so that when you run the container, it automatically applies it. So if you don't want people in the container to be able to use nice access to raw I/O, for example, you just set these to no. No access to syslog. They cannot run kernel modules or they cannot insert their own. So you have fine-grained configuration tuning over 36 million capabilities now. So that's pretty sweet. And so, okay, so there's 36 capabilities, right? Are those extensive of all the things you would possibly need root for, for a program to run? No, I don't believe so. I don't know the answer to that, but no, I don't think so. I looked through the list and I can see where there might— some of them are real specific, like if you want to edit your own text file, you couldn't use the capability for that unless you apply like some Sys capability to Vim, for example, to actually open the text file and write to it, right? Because you're escalating privileges there. But there's nothing for that in here, I don't think. Okay. Well, let's just go through a few of them just so you get an idea, a better idea. In those cases where there is— sorry, pardon me. You're on your own. Where there is— where you don't think there's a capability for what you need, I would just use sudo and specify the deck. You can configure sudo to only limit to the binary and even the arguments to the binaries that only the user can use that binary with sudo if you specify the create a harvest. If you can go that low and that granular, then you can do that. So let's take a few and look at them though. So capchown, make arbitrary changes to file user IDs and group IDs, right? Capaudit, write records to kernel auditing log. Oh, this brings up a point. So when OpenNSM, Dustin Weber, made a point about using auditd in containers to get information on the system, well, you can actually drop those privileges using cap_audit control and write so that I believe this thing will be vulnerable to what he was talking about. In our open initinfo, we're talking about containers. But so that's the capability there. Also fowner, so it's bypass permission checks for following user IDs. The user can then, if the binary has it, the binary can run with the set extended file attributes and be like applied to chptr for example, etc., etc. Lock memory, kill, this was an interesting one. We can actually try this one. You set cap kill and then any user can kill a process. Let's just do that one right now just because it's an easy example. So what we're gonna do is we're gonna, okay, you become root to set the capability of course. So we're gonna do setcap, set, cap and we're gonna say cap kill and then we're gonna do ep and then we're gonna say /sbin/kill. Well, first I want to show you that I cannot kill another process through prior. I know that's been just for demonstration purposes. So I wanna kill this vboxservice n93. It's running as root. I'm not root. I cannot kill it. Operation denied, obviously, right? So we're gonna become this and now at this particular point in time we're gonna set that What is that? Hey, it's you, man. So cap, cap kill plus -ep, so effective and permitted bits. And then what we're gonna do is bin kill and get cap, we can verify it. All right, boom. At this point, this user, I'm not gonna kill that server because that has to do with my virtual machine. I don't want anything bad to happen, so I'm gonna kill another process. Let's do— well, that'll work too, 'cause it's not owned by me. Yeah, let's just do rpcbot, it'll be fine. So that's root process right here at 695. Kill 695. Grrr. And kill. Hmm, what did that work? 695. I don't want any more. I swear I did this the other night. Don't I have history still? Maybe I typed something wrong. Maybe I have the special setting. Oh, I did say that you can use strace too. Hold on, I wanna show that example as well. Here we go, capkill. So here it is, capkill. Okay, so bin kill. Alright, so I'm looking for a process or a PID for that. ATP710. So I used 11473. I killed that. Oh wait, okay, I got it. You have to use the full path in this case. Boom, okay, process is gone. I still have it. Because it has to do, I believe, with the inode. You can only use the full path in this particular case. Wait, I thought that was the— then the bytes were written on that, but you still— oh, actually, I see what you're saying. Um, I guess I don't know. I was saying that sounds like a sample off the top of my head. Um, because we did— beautiful, make sure I love you. Great. Okay, that makes sense. Okay, Brian, you're right. District Allwright said it was stupid. I don't understand what it means for it to be shell built-in though. That means that bash contains the— contains like a version of shell, I guess. Okay. Something that can do the equivalent of shell. Okay. And this is all bash built-ins to make it faster so you don't actually have to run all these programs. Another good thing is you don't have to hash them or anything. Right. And the shell can handle it for you too if you don't have them on your system because not every system has a particular one. So I'm looking at some of the capabilities on my machine and it seems that you can only assign like the EP bits for everything the same, or at least there's no examples where some are EP and some are EPL or whatever it is. They're all EP on your machine? Like you have a long list of capabilities for a certain process and then you only have one set of those bits. I'm just gonna— Yeah, so I guess I don't know if that's just necessary. Like you have a long list. Oh, that's huge. Yeah. And it's just one set of those bits. Is it possible to set different sets or is that just— Oh yeah, yeah. So you can do like if you want to add I in there for inheritable, that then you apply. So this effective and permitted bits are set for all those capabilities that he's putting in there. So yeah, you can set different ones for subsets of the capabilities of the process. Oh, I see. You mean to have like one capability, this particular set and another capability, this set? Actually, no, no, let's find out. OK. All right, so that works for— OK, that explains why that is. Good thing you— I'm glad you— I had no idea kill was a built-in. Yeah, there's a lot of weird ones. I was like, what the hell? All right, is there a way to list all the built-ins? Well, I know there's a built-in called builtin. So maybe builtin has a— Yeah, I have no idea. Okay, yeah, I'll look up later. I'm sure there's a way to do it. What was I doing? Oh, you want to set capabilities. Let's do— I guess one way to do it, you could empty your path and then just tab complete. That might work. Oh, that's a good idea too. I'll try that in a second. So we're doing cap net raw and let's do cap kill, right? And we're gonna use a different set here. We're gonna use EIP, right? So that I think that will test what we wanna test. And I don't need to do that. Oh yeah, I'm, alright, I do need, I'm regular user. Nope, that's invalid. You can't do that, looks like. Alright, let's make sure we don't have some formatting issue. No. Let's try it, let's do it a little differently. Let's do that. And then let's do another one after and see what happens. Kill plus EIP, right? /usr/sbin/tcpdump. Hit cat /usr/sbin/tcpdump. Nope, looks like you can't. It just takes the last one. Okay. And that's actually— yeah, that makes sense. Now I think about it, they're all masked together. So I think that makes sense why that's the case. Okay, so the other cool one was the ptrace one. So we talked about strace, right? So let's go ahead and— oh, by the way, to remove capabilities, use -r and you say, hey, I wanna remove capabilities on this file. You can't remove specific ones, you gotta do the whole thing, otherwise it fails. So watch, let's try this test. So I'll do netcap raw. Don't even need to do that. So that failed. But if I remove the— or even add this, fails. But if you just do nothing, it works. So you actually cannot specifically remove particular ones because of the masking, I guess. It doesn't actually recalculate it or anything. So last thing I want to do is the ptrace, which is cool. And I can actually just go up to my huge-ass scroll buffer and find it. And let's do, uh, what's it, Ctrl+M? Yeah, Ctrl+N. Now if you want to do cat ptrace. Cat ptrace. Work. Oh, sysptrace, my bad. Okay, so I'm just gonna search for cat sysptrace. Hey, Jon. I'm hoping that fell off some region. One second. There it is. Okay, what? Uh, it's pretty much unrelated to the last meeting, so I'm still toying around with one of the things we talked about in open MSM, but for the Bro intel generator, did you get HTML2Text working on your Mac? Yes. Is that a root package? Yes, but the reason it doesn't work is his script only tested for one location. It's actually not in the location I was telling them to test for. So you just have to— I'm gonna patch for them and send it to them. But just change— if you want to change it real quick, you can look for the condition. So -f for the file exists. If you open up the shell script, you can just match that ACL.txt. Just search for that in there and you can just replace it with your path that it's installed in. Or you can do like which, you know, expansion, do which and that, it'll find it wherever it is. Okay, so in this particular case, listen, I am a normal user, right? And I want to run strace against a process that's owned by root, and it tells me that I cannot attach to that process. You don't want to be able to— you're running on a system, and you normal user should not be able to attach to root process, especially because a process 'cause they could have passwords or something in them or something important among other things. So it fails. But if you use setcap /sys/ptrace, by the way lowercase and uppercase are similar in this case. You can use the setcap to get sysp trace to work. And then for strace, I log back out and then I run strace as the normal user again, the baker user, for that same process. In this particular case, I'm able to attach to it, right? 'cause I can see the system call when it executes. So it's sweet. Lucian, if you missed last time, I wanna show you a very quick example of something fun for security, strace. So we did strace, Brian missed it too, you were talking late. So strace, right, so you run strace and then a program after and it just, it basically gets you every system call That's what it is, a system call tracer. It gets every single one and prints it. So basically when you run strace, it attaches to the kernel. It uses a feature in the kernel called ptrace. It sets that bit so that then when that bit is set, every time you enter a system call, the traces record— or when your system call is recorded, every time you leave the system call, the result is recorded so that you actually get the error code and everything. You get the return value. And that's what everything is on the right. All the 40, uh, cover with that. That's one thing on OS X I've never been able to find an answer to is color coding your mouse so that on the terminal you can see it. So all these are zero exit codes for those particular system calls. So, what's a fun one? Password is a fun one, right? So we can actually use strace to write out to a file. What we're gonna do today, we'll just do password. So we're gonna change the password of the vagrant user. Okay, so it tries to load all the libraries, right? And then what it does at the top— we're not all the way up there, but then it tries to open all kinds of locale information, right? So Linux locale information here, Unicode, all the stuff that it needs to properly display your terminal. And then what actually happens is it asks us to enter a password. And actually right now we're stuck on the password prompt. You can see it opens the wrong write system call to enter a new Unix password and actually put it up to the screen. So this one hasn't finished yet. So the strace hasn't finished. It's waiting for input. It just pauses there. So it's gonna read in the file descriptor 0 where you did the password. Some type of my password. Okay, so now you can see they got it, took that, and it put it— now it's asking to retype the password. So my password again. And then I just changed the password and it printed all that information to the screen. So at this point, what system call, does anybody know what system call that's used to actually read in from the screen? It's in the name, or it's in the sentence I just said. Read. Read, woo! So what we're gonna do is we can actually narrow down this. /sys trace -e, password again, I'm sorry, only print read expressions that match that password for Vedric. And at this point, it's just got the region now and we're gonna do password again. And we're gonna do stuff. And at this point, if you're s-tracing, look, you actually got the password. If you can trace this program, you can actually see that the password was— this is what the password I sent to the read system call. So that's kind of cool. I just thought that was interesting. And then the other thing is it actually shows you that when it opens up, we're missing a part, so we're actually opening it's opening up the shadow file, right, to change it. And what it's actually doing is it's recalculating the hash. So this right here, let's just look for open. I think it's open. Read. Yeah, right, yeah, so open shadow for read-only here, but we want it to, should be read-write or something. Password shadow. I don't know what cloneexec is, but anyways, write only. I wonder what nshadow is. Actually, I have no idea what that is. What the hell is nshadow? Oh, 5. In shadow. I don't know, I'll have to look up what the hell in shadow is. That's interesting. But anyways, it actually has to write to the shadow file obviously, so it changes the password. So you can actually see it whenever we look at the previous one with— it's actually— my mouse can't see it again. So this is the actual— in the shadow file there's 3 fields for the password and the first field is the algorithm. And $6 means SHA-256. The second dollar sign over is the salt and the third one is the encrypted password. And strace, you can see how you don't actually get the full system call. It does the three dots and says continue. With strace, you can actually print more of it though using the -s option. So if you want to get 256 bytes, for example, or 500, it will actually print the rest. So now the lines are longer. —why you do that. So strace, you specify a command to run and then trace to it, but is there a way to trace an already running process? Good question. Yes, strace— oops, let's go over to the terminal. strace -p, let's look for process, strace on the root so we can trace— should be interesting thing. Let's trace, uh, dhclient for— so -p 14.1.1. I'm attached right now. And you can see it's just waiting for a select because it keeps looping for, you know, the address and when it has to do that. Can you strace itself? Yes. That is a really good question. Yes. So you can actually learn how strace works by stracing strace. So let's do it because it's cool. Hey, I keep getting 403s on the Git repos. Which ones? Pretty much all of them. I mean, I mean like GitHub or our personal Git server? No, our GitHub account. How are you? Hold on. Give me just a second. Can you see what the remotes are? Type git remote -v or git space -remote or git space remote space -v. Oh, you're using— well, I'm sorry. I don't want to keep— well, I'll just say. So yeah, so at this point we're actually looking at strace, strace, and what we should— oh wait, do we fail here? Yes, because we have to strace something else. So we're going to actually need to do f because we're re-forking, right? Forking ls anyway. So we wanna look for ptrace. So ptrace, okay, so ptrace is the system call that you use to, what sstrace sets so it can actually allow you to work with, get the information from the system calls in the kernel. So you can see how it's used, ptrace, the PID, et cetera. So we're gonna actually look for that. So we're gonna do ptrace -sstrace. Ptrace and then you can actually see we have a bunch of syscall or ptrace calls, right, to actually get that information about the strace process. I guess we don't actually need that for that particular case. Oh, you can't do that anyway. ptrace me operation not permitted. Okay, so I can't do the fork thing. I don't know why, but yay. So, alright, so we got capabilities and we got strace again. That's pretty cool. So I mean it's real cool for doing things like malware analysis, right? If you wanna see what piece of Linux malware does, you run strace to see all the system calls. You can like find out what files it opens, find out what files it writes to by looking for the write system call for example. Awesome. Or whatever, does that write to files as well? I don't know, I'm not sure, whatever. Oops, wrong write. 7, no, where's system call 7?— write to a file descriptor. Okay, that works. Write to a file descriptor. So that could be a file. All right. So that pretty much covers Linux kernel capabilities, which is the sheath. We'll quickly go through— I can just talk about if you want to find out the difference between the masking, which is more complicated. It's complicated, at least for me, but it's probably— it's more than you really need to know anyway if you're just an admin probably in most cases, maybe not all the cases. But so you want to go to sets. Let's look for sets. Oh, there's too many matches. I'll create one just called— I'll just search for it. File capabilities and thread capabilities, that's what they are. So search for thread capability. Alright, so here we are. So this explains permitted, inheritable, and effective. And then for the file ones, explains the permitted, inheritable, and effective. Because I said they're different for each one. So I actually don't understand the permitted, inheritable. I read it a few times and I'm just like, I don't get it. Let's give up. I didn't care that much. So, you know, I was interested in looking at it. Also, you actually, to be able to use the file capabilities, there actually has to be a capability set called setfcap. So just be aware of that. That's not actually enabled, enabled to use it. But what I'm guessing right now is that when you run setcap, it actually sets setfcap. Let's find out. Strace because strace is fun. Net— or capnet raw. Let's pick up tcpcm. -u /dev/tty0. What we're looking for, we're looking for capset, right? -e capset. Yep. Oh, we can look at that. We can get everything that's passed to it. And you can see, look at the very end, set scap. Yay! Cool. Are there any security concerns with using the file superset instead of the— Um, not that I know of. Well, one thing is you cannot drop you can't do dropping from the file system ones, right? Because you run the process in the inode. Like, you can't drop privileges from there. That's one difference that may be a concern if you have a program. The threads are able to drop it. So they can become root for a particular task and then drop it after. So that's kind of maybe a serious concern. I don't know of any others. Not to say that there isn't. I'm sure there are, but my knowledge is That might be something to look into, Jon, and to read more about. I know— yeah. Well, do you know if there's a tool in Linux that you can view the entire inode structure that's in the base? I know you can probably do it with Sleuthkit inode tools, but— I can't. Can't think of any? I can't. Oh, I can't? Yeah. Isn't that part of Sleuthkit? Yeah. Okay, yeah. I was wondering— I was wondering something in the base. Base system. I can install it real quick. Is there a package for Sleuth Kit? Yeah. Let's take a look at that. What is Sleuth Kit, Braylon? It's a set of forensic tools. The bread and butter of it is like viewing stuff in the file system commodal. So there's some stuff in there that will allow you to parse the master file table out and actually view it in its raw form. You can also view partition information. So if you use a utility called mmls, if you run that against a raw image, it will show you the hexadecimal locations for the raw partition structure. And then you can do things like full acquisitions and analyze the IMAP structure for certain things. Oh, do I have to run this on an image? Looks like I do. I was hoping to— basically, I just wanted— okay, so I got— here's the IMAP TCP dump. I just want to be able to print the entire structure. You know if you can do that without an image? I can't because it looks like an image is a requirement here. I was looking for something later. I thought you had an image. That's fair. I was looking for a solution to it the other night. But I can— Do you need an image? No, I just want to do it on my host system so I can like just check out what— talk about what an inode looks like, for example. We want to see those capability bits set in the inode. I don't know. We'll figure something out. I mean, you could create one real quick. Create what? A small image file with dd. You wanted to? Okay. Or we can do it another time. Do you want to do it off the top of your head? I think, uh, ddif— oh yeah, it's a file for git. Equals— you have to make one or give it something that's already on your— dev/null? Or dev/work? Yeah, I mean, it's got to have substance to it. It can't just be dev/null. It can be dev/random or dev/urandom. Yeah, I've set dev/null as the— yeah, okay. It says you're after— oh, this one, sorry. This is just to do this. We need tcpdump file. And then just output file, OF equals my dev. And then do anything special? Uh, I mean, I guess you could specify a block size if you wanted. Just do like bs equals— 2? Yeah. I'd love to see— oh, okay, so— and then I think you could just run— it's just a— well, I think you can just run iCat or mmls against that image file. It's the same exact thing. Uh, I'll just go ahead and submit a different— no, we'll try it. And then now this, I know that this is different, so I'm supposed to do— I know— wait, you said what was the other tool? MMLS. And then .ydoc. Oh, cannot determine partition type. Um. I have— at this point they're the exact same files though. I have another image you could use that I can't push to the repositories. Well, maybe we'll figure out another time. We'll save it. Maybe we can figure out a tool to do it. I can do this for one of the NSM talks next month. Okay. Yeah, I just wanted to see what the capabilities look like and talk to— I know some Why isn't that easier to get? man -k inode maybe, see if there's something built in. What are you trying to do? He's supposed to look at the inode structure. Oh, istat, as in metadata structure? Was that part of the toolkit? Yes, but you need the image in this case here. What is the K flag in man? It's same as apropos. It just searches the manual page. Okay, so instead of just loading man ino, it's actually having it searching for all the descriptions that match the word ino. It's just descriptions, right? Not actual content. Right, it's just descriptions. And it's just to verify that I guess you have to do it in a minute. And actually at the top you can see it says apropos. So the other— let me just pull those. Same thing. Looks like you can use the long— could do long format. So quote apropos searches the short manual page based on the descriptions for keywords. Okay. But you can also do this. And do Inode for example, do this exact same thing. So speaking of that, is that a link? Which type? Apropos. Is hash, okay. Apropos. That's a link to what is? What is what? What is LF? What were you going to say? They're different though, right? I mean, is it just using the executable name? Because doesn't whatis only search the— Whatis database? Or it only searches the name of the— Oh, sorry. That's whereis. That does whereis database. I don't actually know, it's my job. You know what I'm saying, right? Searching descriptions? Like, isn't whatis different from apropos? Doesn't whatis only search like the name of the binary, not like the description? Is that right? Well, they're the same though, at least on Ubuntu, because the symlink from apropos was whatis. Well, some utilities work by they just get the name of the executable and they change the file. That is actually, yeah, that's true. Okay. Yeah, that's a good question. So let's do display what line manual page descriptions. Alright, so if any name matched, right? Manual page names. Okay, so it matches on the name. You're right. And the question, the other question was match with one that's apropos to That searches the description. Oh, you're absolutely right. Thank you very much. Despite the names, it sounds like they do different things. Good point on that. Yeah, so how does that work? It gets like the— it's in C stuff, so I don't know. Is it like it's using the exec or something? It's like in Bash if you were to get like $0. Like you get the name of the script. And it's like, oh, if it's this, then I act this way. If my name is this, then I act this way. Let's strace it. So strace whatis, and then we'll do ls. And hopefully— we want to see— ah, it is exact. Well, yeah. Why didn't you use a -e flag for this one? Because I don't know what system call to look for. I'm not real sure what would be the best. Does it actually search for a specific system call? Yeah. So, hold on though. Exec right here, well that's actually because I think it loads that whatisls. So let's just do whatis by itself. And— oh, it's probably good to know— no, that doesn't— no, that's not— ah, hold on. So strace writes its output to standard error, so you'll have to redirect it. So if I do this, nothing, right? It's already just stdout, so you actually have to redirect it. Just be aware of that. So, exec. Oh yes, I used the exec. So let's just, just to make sure I'm not stupid or anything, let's do this. No, that uses exec too. So I guess they all— oh, I think it's bash uses exec. Never mind. Okay, I was hoping that would have something to do with it. Yeah, as for what system call that will help us to figure this out, I don't know. So what is it doing? It has to open up It would be— what would it be? It would be whatever arguments are passed to test it. And that may not even be a system call. It might be a function. I don't know. I don't really know enough about this program, Jon Schipp, but I'm learning. Learning is sweet. Let's go down further. So at this point, we're— what are we doing? Manual config. Oh, okay, so this is where, this is where, um, what is this working? It's really looking through the— it's opening all the files in the manual database or manual pages. So we need to start something prior. All we need to do, I think, is we need a grep for, um, "What is." Nope. "Prep for apropos." Nope. Well, the thing is apropos is calling "what is," not the other way around. Ah, that's right. Well, guys, at this point I don't really know what to do. So don't you want to be tracing apropos and searching for "what is"? So let's see if you find anything for what is. So what was Well, yeah, you're right. File for whatis. Okay, so that's fine. Alright, user, man, okay, so access that file, it's not there. Read in whatis, whatis, whatever. That's not helpful. I wonder if it actually happens like not in a system call but in a normal, like some function. Right. And then it wouldn't show it. Yeah, I'm not programming, I don't know how that works. Yeah. I mean, how is the thing? Well, how about one of us look it up for the next meeting? That'd be really sweet. Would you be interested in taking on that? Sure, I remember. Yeah, so there's another program called Ltrace, and that actually does functions. I've never used this one though. I'm actually Let's just try it. Oops. ltrace. Oh, looky there. Oh, libc*. So let's— damn it, this is pretty sweet. ltrace apropos ls. Oh dear God. Alright, so let's start with— does that go to— let's see if it goes to this. No, okay, so that goes to SDGRP2, and then you want to do add— I don't know, I'll give you 1,000. Oops. Hopefully it's somewhere within 1,000. I'll try to learn Ltrace for the next meeting, I think. Wow, this is really ugly. Can you convert that to whatever it is? Yeah, that's just got to be a way to convert that from text. Uh, asky, ask. Don't see you guys. Oh, okay, well, I'll save it for another time. Yeah, I really want to know how that works. Yeah, I feel weird talking about it because like it's never something I've read about, it's just sort of something I noticed and like sort of explored a little bit, I guess, because like I noticed a lot of things work that way. Yeah, I, I I've noticed that too, but I guess I've never really thought about it. Yeah. It's kind of like— well, I don't know. I guess the first thing, because I'm a Vim nerd, I guess the first thing I think I noticed it with is how— what is it? I think Vim is similar to Vim.bat. How the hell does that work? Yeah, yeah, yeah. That's right. I don't know. I think grep's the same way with fgrep and egrep. Yeah, probably. Yeah. Let's do that. Am I too gremlin here? No, that one's just— wait, why did that— oh, gremlin, what the hell? Nope. I thought they were. Yeah, I thought they were too. Oh, this is BSD2, so they're Um, let's try here. Which— oh, okay, I guess they're not. Oh yeah, that was weird. Actually, I remember this. I was actually looking at the same thing, and I was like, why are these all like huge executables? Like, why did they— if third party has the option to do that, it's weird that they do it. Like that. Right. But maybe, who knows, maybe grep -Kf calls the fgrep. That's— I honestly have no idea. I wonder if there's something in the man page about that. Because it goes like— Because I'm pretty sure— okay, so yeah, I did like a diff on the binaries and they're all different and they were slightly different sizes. But they're different. Okay. Yeah. Um, SimLink to change— how do you word that? Change— SimLink change— Change the program? I mean, because really it's not that it's a symlink. I think that it's just the name. It's the name. Like if you were to change the executable name or the actual binary name, I think it would actually change it as well. So it's, I mean— Oh, so that's stored in argument 0. Yeah, that's what I'm saying. I think in the program there's just a case or switch or something for argument 0 and then do this. Okay, that makes sense. Damn, I wish I knew how to use a debugger right now. Efficiently, we're gonna be here forever. I'm loading GEP pretty soon here. Um, test argument zero. Well, that concludes the one. I'm going to go ahead and shut off the video. All right, thanks for attending here, right? We'll follow up next week here. Stop sharing. And