LUG Meeting (Docker, Quiz, Sudo,) 01-26-2015

Alright, cool.

So there was some talk about the name. Do we want to address that tonight, kind of have a consensus or not?

I don't know, guys.

So just to recap, LUG was the original name, Linux User Group.

Yay, LUG!

And then since we were already getting Richard Stallman, he recommended, or he mentioned, which is correct, to say GNU/Linux because Linux is the kernel and GNU is the tools that created the operating system, that have the operating system together. Together, which I'm fine with that. And then I introduced the idea to add another U to mean Unix, so GLUG, G-L-U-U-G, GNU/Linux and Unix User Group, because I was thinking maybe we could talk about more Unix-like operating systems or proprietary Unix operating systems. So there'll probably be less talk or less interest in that unless you guys are— I don't know how you guys are about that, but So it's not a key thing, but I don't know, does anybody have any opinions on what they want to—

I think we keep it simple.

I like log. Yeah.

Because we got freshmen and stuff, and it's just too confusing when you start having, what is it again? And even OpenNSM people are like, what is it called again? OpenSMN? So if you're just like log, people are like, OK, log, I can remember that. L-U-G, yeah, Linux Users Group.

OK, that makes sense.

And then they get in, and then we load them up with all the information about GNU and GNOME and systemd and all that dirty, dirty stuff.

Does everybody agree with that? I think keeping it simple is the best way for people to remember it and advertise it.

OK, cool. Moving along. Seth Dory.

They're probably shooting pistols while I'm talking, so I'm going to be a lot slower. So, a few things. Docker 1.4 was released.

And honestly, I looked at this, but I don't remember what the additions were or the fixes.

Let me quickly look at it here. 23 vulnerabilities. The advisories are listed at this link. Oh, they added the Overlay File System.

It's a new storage driver.

See what else we got. More security stuff. But they did add, in addition, they're working on some new tools. They have Docker Swarm, which is a clustering tool, which we briefly talked about in the prior meeting, but I haven't had a chance to play with it.

So there's Swarm, Compose, and Machine.

Now, is this for coordinating apps that run across multiple containers then?

Yeah, yeah. I think Swarm is for the— or composes for that. It's one that brings them up if they have dependencies. And Swarm is a way to—

just quickly, just to make sure I don't have it wrong— distributed applications.

So it's clustering and failover. Okay, so you can have— you run Docker on multiple hosts and then have your app or your container run across those.

So Swarm is—

looks like that's what they're trying to solve. And the other one was Docker machines.

And this is—

log into the host and follow installation. Ah, it's a quick way to bring up VMs or provision your system for Docker. So it gets you running right away. So you can actually use a provider. You can use virtual machines in that case. Or you can use AWS, Azure, VMware. You can actually use that to spin up a Docker machine for you and you can actually and then build your containers with that. Which, when I get some time to actually play with these, I'll talk about them. And if anybody else has experience with them, they can talk about them too.

That's cool stuff.

What else we got? Vagrant push. Oh yeah, I didn't have a chance to look at this one. So Vagrant's got a new command out here. Okay, so it is a single command to deploy any application.

And let's see how—

that's 3 minutes, I don't want to watch all that.

So define push FTP. Ah, okay.

So you can actually push these to services. So the FTP one was a simple example. You're actually pushing the Vagrant information to an FTP system. Heroku, right, which uses containers. It's a way to serve your application.

A lot of businesses will use it for their web applications.

And you can actually deploy your Vagrant machine directly to Heroku in that case. And Atlas, I'm not familiar with Atlas. It's a commercial product by HashiCorp. Does the serving of machines as well. So that's something that's new. Also a new one, Vagrant— let's do Vagrant global status that came in the recent version. This actually tells you all the machines, all the Vagrant machines you have on your system. So you can see I have this many that are actually up. And you can see I have—

I'm going to make that font quite a bit bigger here.

And let's kill one of the whites. But you can see that I have two VMs running now with VirtualBox. I have two Ganeti nodes. So this is a way to actually look at all your current running configurations of Vagrant. It's actually beneficial because before you had to go to each directory and type vagrant status and look for the .vagrant folder in there to get the status and see if it actually was there configuration. Is anyone here not familiar with Vagrant?

Okay. You?

I've seen it before, but I never got it to work.

Oh, okay.

So it's a really cool tool. We're actually using Vagrant to build our virtual machine infrastructure. And I use it at work. We use it to— before we deploy— so essentially it's a wrapper. It's a Ruby wrapper around the provisioning of virtual machines. So you create a config file that tells you how to bring up the virtual machine, what operating system to install.

Like, you give it an ISO or a Vagrant box or something.

And what will actually happen is it'll read the file and start the provider. Vagrant connection will work with VirtualBox, VMware, and a few other ones. And you can actually type vagrant up and it actually reads that file and will build a virtual machine for you right there. And one cool thing about Vagrant, it has the ability to do provisioning. So you can call a shell script, which is one of the provisioner types, and a file to actually copy files from your host system to the VM. Or you can have it, hey, after the machine boots, run this command or run this script to actually install all the software that you want. And the idea is you just create a text file, like a 100, 200 kilobyte text file or whatever, or depending on how much stuff you have, and you give it to your developers and it solves the problem of, well, it doesn't work on my machine. So if you're developing an application and this guy doesn't have the right library or some version's off, it doesn't work on his machine, he can't test the stuff, you give him a Vagrantfile send it through email or put it on Git or something and they tech baker it up with that file in there and it builds the machine with the exact specs as the person that created the file so they're the same. Therefore, they can actually use the similar software and actually do their testing. So at work, before we even push out a virtual or build a new server, what we do is, for example, I'm working on Cuckoo Sandbox. It's a malware analysis system. And before I even try to dick around with the hardware, install the operating system on that and configure it all. I want to do it in a virtual machine because it's easier. I don't have to go messing with that, plugging all the cables, breaking it if I mess up the configuration because I'm learning as I go. Then I can actually just do it in a VM with Vagrant and quickly spin up these things and practice, try new commands, edit the configuration file, bring up another one. I can have this development environment for this. And I can give you a quick demo. It's really cool. And we'll do a future Vagrant talk if you guys stick around for those that haven't seen it before. Before. So, if you're running OS X, there's a package. If you're running Windows, there's a package. You can run Vagrant on Linux as well. You can go to repos, Vagrant. This is mine. I have these all— they're all on GitHub. And can everybody see that all right?

Yes?

All right. So, for example, Sagan. So, let's take a look at this one. So we're going to look at the Vagrantfile. This is the file that you pass around and it's just a Ruby-like file. That says, hey, I want to configure a virtual machine with a box of trusty. So Vagrant has this concept of boxes, which are prepackaged VMs. So if you want to have an Ubuntu box or you want to install the Ubuntu operating system, you actually have to have the box. But luckily, Ubuntu and many other Linux distributions will actually have the box available online. So you go to cloud-images.ubuntu.com and you can download their Vagrant box. And so when I type vagrant up, if it doesn't find it on the system as labeled as trusty, it will go out to the internet and download that box pull it to my machine and then start the box. And the virtual machine— I say box, I'm referring to virtual machine anymore— but you can say, hey, I want the virtual machine's name to be Sagan. I want it to have this much RAM.

I want it to have utilized this many CPUs from my host.

And at that point, once the box is up, it actually runs your provision stuff. You can say, oh, I'm copying the source file, Sagan.upstart, to this file on the host in the home directory. And then eventually after that, what I do is I run a shell provisioner to run this provision.sh script. If you look, we get out of here real quick, these files are right there. They're in the same directory. So there's my provision script right here. And it's just a shell script that says, hey, install my stuff.

So I just give it the list of packages that I want installed. I wrote this script.

But you can use any language you want. It'll run it. So Python or whatever else you want to run, as long as the script tells you how to configure the machine. So now literally I type vagrant up.

Vagrant's—

let's actually see if I have that box ready. So Vagrant status in the directory. So you actually just do make dir, you create a new directory and then you type Vagrant in it and it creates a blank state for you. But you can actually play with mine if you want. So I created it so I can bring it up.

Vagrant up.

And what's actually happening is it's gonna look for that provider. It detects that I'm using VirtualBox. Now I was importing that trusty box. I already have it on my system because I downloaded it in the past. And it's actually gonna go ahead and now it's done. I already copied it. Now it's creating the network configuration. It set the VM, the hostname of the virtual, or the VM name right here as Sagan.

Setting up ports, port forwarding.

And now it's actually configuring it ready for SSH, so I can SSH right into the machine. And it already has an unsecure public/private key combination that's already, all machines will get that by default. So you can just type vagrant ssh and you're logged into the box and you can actually begin utilizing it.

So can you talk a little faster?

No. Yeah, yeah, actually I can. Hey, if I'm too fast, let me know.

I can slow down.

Questions, just throw them out, I'll answer them. So at this point, we actually—

I have a question actually, like up at the top of that file, like what were those files that you were writing to places? Like the, the file, the Sagan?

Yeah, so these file, the file commands, uh, like right here, they're actually—

it's not that, like it was inside the the file.

The Vagrant file?

Yeah, the Vagrant file. Yeah, yeah. So all of those provision files, like what is that?

Yeah, that was what I showed you. So what's actually happening is it's taking a file from the host system and can copy it to the VM for you so you don't have to.

Oh, okay. It makes it easy.

So in these cases, I'm just copying configuration files that I've already made and I know that work. Okay. So I just copy them to there and then when I do my provision script, I say, oh, get it out of the directory and copy it to the appropriate /etc directory, for example. Because you can't actually copy to a privileged path. You can only copy to the user home directory.

You can only clone into a home directory.

Yeah.

That's the same as Docker in that way, sort of.

Yeah.

In your Dockerfile, every line is a line from the home directory or from root directory.

Right.

Right.

But you can— yeah, that's true. That is true. So you have to specify—

I can't cd because the next line will just be—

That's exactly— that's accurate, yes.

So you can see what actually happened was it ran those file provisioners right here, all 4 of them. And then, so there's the 4 right here. And then it went to the shell provisioner. At this point, here's the shell provisioner. And that's where it's running the shell script that I wrote. It's installing all the packages for me for Sagan. And so it configures Sagan so that I can then do log analysis. So now I'm pulling from the Git repo. So we're gonna automate all this.

The cool thing about this though is that you can just utilize other people's machines.

You can go to github.com and you can type Vagrant and then, I don't know, MySQL probably. If you want to find— like, seriously, someone has a Vagrant LAMP stack. If you don't want to take the time to build it yourself, just clone this guy's repo and type Vagrant up and you have a LAMP stack.

It saves you time.

People have done the work already, so they just share these files online.

That's sort of awesome.

Here's one with Puppet and MySQL. I highly recommend you do this. It saves you a lot of time in development. There's tons of stuff. So you can probably have it for web servers. There's probably a Docker one if you want to play with Docker, Vagrant Docker. I imagine someone's created some Docker ones. Looky here. This one's for the boot2docker script.

WordPress.

So you can do all this stuff. This one's running MeezOS. CoreOS is another one. I know that CoreOS actually has their own Vagrant repository. So you can actually try out the CoreOS operating system.

How nice is that?

You literally just clone any type of Vagrant up in that directory. As long as you have the provider like VirtualBox, it'll actually do all the work for you because these guys have programmed it to do that. So I highly recommend checking out Avegraphe. I don't want to make this a whole Avegraphe talk, but it's a very cool tool. If you have any questions about it, you can ask me. So let's get to more of the news. We have Applied Crypto Hardening. This was a website that its goal is to give more practical information about cryptography to administrators. And so they released this document called Applied Crypto Hardening. It's a PDF file. Essentially, it tries to teach you how to configure your servers with, say, SSL or TLS in the best possible way that is known for security.

And they—

so they go through all this. They got examples on OpenSSH, Dovecot, Postfix, XM, NGINX, Apache, etc. So all these— just a guide on what you should do, how you should configure your machines, VPNs. Tells you what ciphers you should use, which algorithms that are known to be the best. So it's a neat little project and we'll keep updating this file as more and more people contribute to it.

And you can see there's a bunch of command outputs so you can actually do the testing and try it out yourself.

Secure Shell, this is an article written that got some popularity. It was criticized from Josh and the OpenBSD team for being slightly inaccurate, but it's pretty cool nonetheless. So essentially Der Spiegel, the German newspaper, it's a well-known German newspaper that's known for its high quality standards of journalism. And it had an article that was written by a guy. They had some Snowden documents that released some information about how they were able to decrypt some of OpenSSH and you can read that. You can go to the article. At least it's quoted from the person. They don't have the details on that in the article anyway. So this, based on that, they actually created— this guy created an article to talk about the cryptography of it and what settings you should have to try to avoid these situations where the NSA could potentially compromise the cryptography. And it usually has to do with weak ciphers. It's usually all as simple as that, down to that. So it tells you how to change it all, etc., etc., like here's the algorithms you should use, etc. But it's a good article and I highly recommend reading it and trying it out. Maybe we'll cover each one of these in a future SSH session. Tyler Manson linked to this article which actually looks really good. We were talking about OpenBSD and FreeBSD. This guy came out and wrote a security comparison between OpenBSD and FreeBSD and it goes into detail. It goes into stack protection like canaries, ASLR, et cetera, et cetera, write execution on the memory segments. And so on and gives you some detail even with like the actual system calls that they're making or the functions, etc. So if you're interested in that, I highly recommend reading this. I haven't got a chance to read all the way through it yet, but little pieces here and there, but it's actually pretty detailed. So if you're interested, and here's a little graph, a little chart that actually shows some of the difference where things are implemented and where things are not. And unfortunately FreeBSD is behind on a lot of things such as automatic stack layout randomization. But I think there is a patch for it, but you have to apply it to the kernel yourself, for example. But yes, if you're looking for a secure operating system, yes, use OpenBSD, but it's a pain in the ass. I have been to some OpenBSD machines in the past, mostly firewalls, things with PF, but that's what it's for. It's designed with security in mind, and with that comes it's very behind with modern tools.

A lot of stuff you can't install in the operating system.

Because it's very slow moving. So it's just something to be aware of. But FreeBSD is a little different. It's a more general BSD operating system that takes the best of all the world— well, I shouldn't say all the world, but it allows you to— it has a large user community and large port system. OpenBSD does not, so it's harder to install the packages you need.

So there's that.

Education epoch, anybody got anything? Any cool articles or programs or courses for teaching Linux or anything?

I didn't have anything.

Wait, I think a bunch of—

oh, never mind. That's Linux, never mind. Okay.

Never mind.

I was thinking security stuff.

So proc period is the next section. This is where we talk about the proc file system, which is the kernel file— this is what the kernel uses or creates that allows you to interact with the kernel for in user space. So since we're in this VM of the operating system, I can type vagrant ssh. We can actually go and have a shell inside of this virtual machine I just created. See, I have my boot system that I brought up with Vagrant.

I'll become root.

You can see that if you look at mounts, you'll see the /proc right here. All modern Linux distributions will mount to the /proc filesystem. And if you go to /proc, this is actually an insight into the kernel. So all these numbers on the left-hand side is a process and so PID 1 for example is your init system. It's the first process on the system. These are all directories so you can go and get information about particular processes.

And you can also query, this is how you directly query the kernel.

One interesting thing about this actually is there's a lot of information instead of actually making calls or you can actually directly read and write from these files to change system settings like even increase your screen brightness if you find the right proc file or something in there. Or a kernel setting that you can turn AFL off, for example, in the proc file system if you didn't want to have automatic synchronization on, etc., etc. But a lot of programs actually read from these files. I wrote a tool called PPS, uh, uh, just sort of packets per second, but it's a network tool that actually reads from procfs, and what it does is it gets the latest kernel statistics on network traffic and then it'll read it every second and then tell you how many, which your bits per second are and the percentage of loss based on your the link speed, like gigabit for example. So that's actually pretty cool. And so that's Proc FS. We can show you an example. One is load average. This is actually what tools like Uptime and top use to actually get the load information from the kernel. So we can actually test that. We can do strace. We type uptime. I'm just gonna grab grep for a load average. You can see that this actual program—

oh, this writes to stdout. Or stderr, I'm sorry. There it is.

So you can see whatever this call that was made actually openproc load average. You can see that top does as well and a number of other tools. So you can actually see that they— programmers will actually take advantage of this kernel facility. And so that's one. Another thing is I think on the lpick1 test The Linux Professional Institute certification level 1, they had a question on there about the procfs and they asked how do you get, what is the file inside the procfs that actually tells you the kernel information, which is probably what uname reads from. And it was cat command line. This actually has, oh actually that's the options you actually pass to the kernel. And actually that was the question. So you can see when the kernel was booted, it passed the VM Linux image, the UUID for the root file system and these particular values for the console. That's one way to see how the kernel was booted. That's actually really useful if you want to see what it was doing. In case a certain module will not load or something like that. So it's really cool place to play around with. Of course every PID, every process, for example my bash process 25656 actually has a number, you can see it right here in the current, in the proc file system right there. I can do cat that and you can see all the files are in that directory. And you can be like, oh, maps actually has the memory layout of that particular process from the stack to the heap. You can actually see where things start and stop. What else happens is, there's another interesting one, procenv is a cool one because as an admin you'll find, or even in security, you'll find a lot of cases where you wanna see how variables are set so that it determines how a program actually reacts based on those variables. And you don't know how it was started so you can actually query the PID number and then envirn for the environment and actually see the values that were passed or that were set when the program was ran, which is really useful. So that's just a quick intro to that. So in this example, we already talked about the load average one. This one is how you can use ProcFF to detect whether your hardware or your processor is using Intel Virtual Machine Extensions like the VT chipset. And we'll actually have to have a machine that has that permit for it to work, but this is a VM.

I don't have it passed through, but you can see that nothing is returned.

But proc cpuinfo actually has information on your CPU. So this is one way you can get it. It tells you for each core, there's only one core in this system, so it's just this one group. But you can see the genuine Intel is a Core i5 at this particular frequency.

Etc.

And the flag is what we're actually looking for. There'll be a particular flag if the VT chipset is enabled, whether it's AMD or Intel. And so we're actually looking for BMX for Intel, SVM for AMD.

And damn, I missed it.

So you can actually see for every processor in this one we had a match on the line. And in this case, grep is already— anytime it matches something on the line, the entire line is highlighted. So you can always really need to see that. So we can add—

let's do this again. Let's add -o though.

And that will just match that particular part.

Is that—

oh, it's 'cause—

never mind.

That's 'cause we're doing a beginning of line match.

That's right.

But you can see that these proc— these are cores, each one. So we have nproc is 8. So there's 8 cores in this system.

So if you count— and we have 8. You can see if you just counted those number of lines.

So let's just go to /lastproc CPU info and we're gonna look for VMX and there's the match right there. So it just basically says that this particular core, core 0, has a VMX. This next one is core 1 processor and you can see that has it all the way down. So that's one way to tell if your machine without having to look in the BIOS, without having to look it up online, whether it supports the kernel VM extensions or the hardware extensions. Okay, and then that takes it for proc period, SSH section. So the fast— if you want to transfer files, Waylon and I had this experience the other day. I was at my house and it was slow. The fastest way to transfer a file in SSH is to use the one with the weakest crypto, and that's the RC4 cipher, or RC4. So you can pass the -o Cypher equals R4 option to copy a file with that. You should also use, if you're copying a file and it's large, you should always use compression if the network link is slow. That way the host actually takes care of the compression and then in the case where the network link's slow, that's okay because you're sending less data because it's compressed. But if the host is slow, that won't help you much.

But you probably don't want to do this over multiple networks.

I mean, the NSA is completely broken.

They're C4, so.

It depends, what do you mean by that?

I mean, it's weak encryption, so this is fine if you copy things up on like a local area network.

Oh, right.

So let me amend this and say this is for cases where the data doesn't need to be confidential.

Yeah. So in this case, I was already connected to the VPN.

So you don't need extra encryption, right?

Yeah.

Right.

Like if you're copying an ISO image or something of a bunch—

you know, something that's big that you just want to get to another machine, it makes sense to do it. For that. Hey, ARC4 is an encryption scheme, it's not—

Yeah, that's what I'm saying, like this is not appropriate for every situation.

That is correct.

And then with SSH you have different levels of compression and FTP is using GZIP compression, so we can do SSH config, Compression. And you can say whether to use compression, yes or no, and the compression level. And you can see that 9 is the best compression. That is, it's the slowest to actually produce, but it is the best of them. And it uses the same compression algorithm that gzip does. You can set these in your files, or you can set it on the sfx command line with the -o option and compression equals whatever. Does anybody have anything to add? Does anyone want to talk about anything? I'm just kind of going— yeah, go ahead.

Yeah, I am curious. So what's the difference between mounting it like with SSHFS versus using NFS to use SFTP?

Well, SSHFS uses the FUSE file system, right?

Yes.

So I mean, it's just two different ways. I mean, Have you used both of them?

I have. Back when I had Ubuntu, I used Nautilus because it was built in. Now that I'm on Gentoo, I mount it as a FUSE directory. But I'm curious if this has any real performance differences.

I do not know. I've never tested that.

That'd be something to be worth looking into.

There'd probably be a paper in that.

Yeah, could be.

Or if anybody has an article online. Yeah, if you care to look that up and see if anybody has it, if you want to. It's up to you though. I'd be interested in seeing it too. But I would expect that SSHFS to be slower because it also has to work with the FUSE driver. It just seems like there would be more operations.

All right. That's just a guess though.

And are the flags for the SSH— are the flags for SSHFS like the same as if you're doing SSH?

I haven't used SSHFS in a long time.

So there are flags for it.

Do I have them still here?

Oh, I do have them.

Yes, they have the mount options, -o. Now those won't be the same, port's the same as in SSH. Let's say a lot of them will be different actually.

Oh, wait, execute command, okay, it's actually command instead of SSH.

Let's look for compression for example, arc-compress. No, there's nothing matching for that.

Oh, there is.

'Cause there is one. So -c, that's one we can try on compression. What was the other one?

Level maybe? Or did we have compression level? Is it gzip? No, I don't see anything.

So it actually looks like they're very different in terms of options. Can you do ciphers? Let's do alg or algorithm.

Nope.

Oh, yeah, so this doesn't look like it does anything unless they use a different word for it. Okay, well that shows a huge difference between the two then. So SFTP will have all those options too, but SSHFS does not, it looks like.

Alright.

So this is cool.

Make temp.

This is a portable way— well, I think portable except OS X. It just works. You can laugh on that one, Shane.

I'm gonna get it.

But to create temporary temp directories. So it actually just returns a temp directory.

So if you make temp here.

Oh wait, yeah, this one, let's just do it on Linux. And it created a temp directory right there. And you can use this in your scripts if you want to have temporary places to place your files. And a lot of programs will call this because it's a good way, it's a smart way to do it. One, it's almost completely portable across any operating system.

And it gives a unique directory where you can copy files to a temp directory.

So you don't have to interfere with anything else there. So we can actually see that that was the result and we can actually go to that directory. So TMP.

You know, here. I am. Okay, so there it is.

So it's a way to do things like that. So you would use it in your scripts to create a directory and do stuff with it.

So pretty neat.

I've seen this in scripts people write. What about shell space?

Suspend.

So this is a way to suspend your shell. And so you know how you can suspend a program with job control, right? fg, bg, send to the background, like for example, ef puts it in the background where I can still type commands. You can't really see it because it's going so fast, but that foreground and quit it out. So we're using—

let's just do a better example than that.

Let's do top. Top's in the background right now. And if I want to bring it to the foreground, Ooh, looks like there's a TTY issue with that. Help me out here.

What's something else I can use? Something that just pops? What'd you say?

gedit?

Oh, I don't have gedit on this one. Or Sublime or something? Pops up a window?

No?

Wireshark? Wireshark?

Why is it so complicated?

tcpdump. tcpdump works. That's a good idea. I don't have tcpdump on this one, or tshark on this one, but this will work.

So we're listening on this interface. Alright, so now it's actually still printing but I still control the terminal, right? So I can send the clear signal and it actually clears the terminal. Type fg, it's back, right? Now I have control and type clear, it's going to be— boom, sending that to the process itself, the shell. So that's why you can do job control, fg and bg. And so the ampersand at the end is how you send it to the background right away, right? So you can actually do that with the shell.

You can actually send that.

So I still don't work right away.

I thought ampersand just ran it in a separate process or something from the shell.

No.

We can look at the technical definition.

Or let's use this.

I think you're thinking of exec, which is kind of the opposite of that.

Yeah, that replaces the process. Let's see what they got in the Bash Man page. What would that be called? I might look for background. Yeah, so it does. Okay, so you're right and I'm right. It goes to the background. The more technical definition is ran in a subshell. So another process is spawned.

Okay, cool.

Thanks for bringing that up.

I actually didn't think about that.

So that's interesting. And then, okay, so suspend. So we actually cannot do this with the login shell. So what we're gonna have to do, we can do, I think we can do this and then suspend it because this is no longer a login shell. Because we invoke bash without having the --login option which is what the system does when you log into the system.

So in this case, we're able to suspend that particular process and we can actually bring it back.

Right, so echo, or what is it, SSL.

SHLVL.

Thank you.

Ah, dash shell level.

Alright, so we're in level 2 now.

Suspend it.

Echo sh lvl.

We're now in level 1.

Zoom back and we're in level 2 again.

So that's one way to do that.

But we can actually suspend the process with the login shell process with the -v option.

It forces it.

And now it's actually—

wait, am I still in level— nope, I'm not.

So it looks like it actually put another— oh, it actually dropped me back down to the previous shell before I used sudo -i, which actually invokes an interactive shell. Previously as I was. So that's really interesting. And like, now I'm back as root.

So that's kind of cool.

So instead of having to maybe exit out and lose your current session, right, with the history and writing out the history and all that, what happens when you exit the bash shell brings it, you can actually suspend it and drop back down to the previous shell.

So that could be useful, you know what I mean.

The only example I can think of is for history though at this particular moment because when you exit the shell it actually writes out the history pen stuff. But if you want to keep that and not run it at that particular point in time, you can use suspend.

So— Or—

Is there a way to use Docker container with that? Like push the Docker container like to the—

What do you mean exactly?

Like, uh, push it into the background? Like get a shell from a Docker container and then jump out of it?

Yeah.

Like leave it running, but like without, you know, I know you can do Ctrl+P, Ctrl+Q or whatever to like Quickly.

I don't know if— I don't think it'll work, but we can try it.

Let's try it.

Fuck it.

Um, let's go. Let's go to— do a live, right? It's a big budget show.

We do what we want. Speaking of which, uh, I was trying to— I was going to try to mention this earlier, but the ACM open house is Thursday at 7, even though you guys already found the best SIG on campus. You should come and get free food.

Help represent.

So you find out about the other best group, which is right before this one in the same place.

Yeah, yeah, if you're interested in intrusion detection, malware analysis, and security defense, we have a pretty cool meeting where we have a lot of industry speakers come and talk.

So famous people.

Yeah, we just had one that wrote— where's that book right there next to Devin? The Applied Network Security Monitoring book. Like, the author of that—

both authors, both authors of that just gave us like a basic overview of like, this is what flow analysis is and these are the tools that you use and here's how they work.

Yeah, that's pretty cool.

Yeah, it's pretty amazing.

You say pretty basic?

I said pretty amazing. Oh, okay. Amusing?

No, amazing.

Good job, loved it. Yeah, that's 10 out of 10, would fuck again. Oops.

All right, so Docker run. Alright, let's try this. I doubt this is going to work because I know, I'm sure it's not as simple.

We don't even have—

okay, so suspend.

We'll find out.

What does it say? Oh, yep, it just dropped me back down. That's interesting. What level of shell was I at prior?

You can go Ctrl+P, Ctrl+Q, I think will get you to actually like leave this running and step back out of it.

Ctrl+P, Ctrl+Q?

Yeah. Nice, I actually didn't know that. Cool.

So at that point, let's go back into it.

The way you have to do that?

Like if you have a controller running and it's got, you know, if you've got bash running in a container, if you hit Ctrl+P, I think, and then Ctrl+Q.

Yeah. Okay. All right.

So that doesn't kill it, it just, you get out of it and it's still running. And so you can then attach it to it again and get right back where you were.

So in this particular case, it does not drop us back out. You can see that we're at shell level 1. We type suspend and then we echo again.

We're still at shell level 1. So in this case, it does not work.

What happens if I pass the -f option though? Yeah, so what I suspect is happening is that init is just spawning a new shell potentially ready for us because the Docker system is a little weird because it's not completely the same as the way Linux is used to do. That's just a guess though.

You could test that with dollar dollar.

Got ya.

Funny how they call— well, I guess that's right.

Process 1.

Exactly right.

So I think that's right, isn't it? Suspend.

So $1. Yep, we're still 1.

So that's exactly why. And it spawns that new process for you. What's $1? It's the PID of the—

This is the best way to test shit.

What?

It's the PID of the shell.

Yep. Okay, so that's that.

So another one is group command. So this I actually, I'm a little embarrassed, I've only recently known how to do this. So I write a lot of shell scripts and it's been a pain in the ass. Sometimes I like to do one-liners and when you don't have a way to group the shell commands, I'm not talking about subshells, I don't actually want to send all the commands executed to another shell. Which is a sub-process of the particular shell that you're currently in, it's nice to be able to group them. I couldn't find a way to do that. This is how you use the curly braces and you can actually group commands in this particular way. Let's see if we can give an example if I can come up with one off the top of my head.

So we're going to do true, which is exit 0 essentially. So true and echo yes or more. User. Let's do echo made it here and an ls.

And you can see what actually happened here is not what I wanted to happen. It actually executed true, returned true, so then the ampersand, or this one is an and operator, so then it executes echo yes And so you see them printed. But then also happens, it actually, we want it to be do these things or do these things if one of these things fails. But what actually happens is it does these things, skips this one, and executes this one. So in a lot of shell scripts, it makes, it's a pain in the ass to actually have to group this in different, or to use subshells, which I always use before. But that's not a really efficient way because it spawns a new shell process and runs those commands and exits. But in this particular case, you can actually just use groupings.

So we do this, we do this, and we execute a semicolon.

And now that actually did what I wanted. It executed the first two commands and then that's it. It didn't evaluate the other ones. So doing the reverse, false, this actually returns 1, so it will fail and then it will go to the OR because the AND did not match.

So you can see it's— this failed.

It returned 1, so this one didn't get executed. And then it immediately went to the or section and went and executed, made it here, right? And then what actually happened was that it also executed ls, which in this particular case is fine.

We want that.

But in the former case, it didn't do what we wanted, right?

So that's kind of the difference here.

So that's a fun little way.

I think I have a few examples where I use some scripts to make things a little bit neater.

Why do you need the semicolon?

That's required. I'll just show you. It's just you get a syntax error otherwise.

So I think Bash treats the close semicolon as like a separate command.

Yeah, that's right.

I thought there was a— see, it still wants more input. So I don't actually get a syntax error. But it wants more information.

So, yeah.

So if you actually didn't use the AND, what you'd actually have to do is make it here, this, and then you want to do ls, you have to use two semicolons. If you do not want to do some sort of operator that will do, yeah, the AND, you can do that as well. And you can see here you got both of them still. So that's one way to do it. And that's actually very useful. So I highly recommend if you write scripts and you don't need to do the huge long if clauses, right, if then else, you can do little things like this just with the or and the &&.

So it's really helpful.

That'll make you a better shell programmer.

So let's do— Let's match directly. Let's see here, not exactly what I wanted.

I want to match on the same line, so let's do Mm-hmm.

Use a little better with that. Let's do rsh. Really? Hmm. I wonder why that didn't— let's try that. Oh.

Anyways, I wish TrustSparkle thought I could get that matched.

I don't know why they didn't do that. Let's see here.

I'm just trying to give you an example of what I've used to do that.

Oh, here's one right here. GR security. So this is—

I wrote a vagrant configuration to provision the GR security patches to the Linux kernel, which improve overall security by providing discretionary access control and improvements in ASLR and more stack protections, for example. And that's actually ugly.

Well, let's do this.

Okay, so we want to look for that line that I had with the curly braces.

You can see it right here.

So what we're doing is we're making— we're running make install to install the Linux kernel. And then what we're doing is we're running this function after if the Linux kernel returns— the install command returns 0, so it actually correctly installed. Then what will actually happen to the grouping, if that does not happen, it will die. And then it will exit out and print this error message, "Failed to install new kernel," which is just a shell function I wrote.

Excuse me.

So this is an example of how I do that complete, that grouping without having to do a bunch of tests for different exit codes and such. So just an easier way to do it. I can fit it all in one line.

There's an example of that. Alright.

Vim vicinity.

Oh, does anybody have anything to talk about?

Anything at all?

Kind of.

OK, pretty good.

What's up? I just found out about this thing today called Script and Script Replay.

Have you heard of that? Yes!

I never knew about that.

Yeah, so that's cool.

So you want to demo that? Sure.

OK. Yeah, do you want to use my machine or do you want to use yours?

Well, mine's a phone right now.

Yeah, use mine.

It's got Script on there.

OK.

Which version? Do you have the—

well, I'll just talk about it.

It's gonna be the old one with the—

BSD. Yeah. Make a new version.

So just a little—

while he's getting ready for that—

I actually use this when I'm consulting and my client wants to know a record of all the commands I type and all the output, standard out and standard error. This is exactly what it does.

It records the entire terminal session to a file.

Memory serves, it also has a usage when you're exporting and then going to elevated privileges or switching users or something.

This program?

Scripts?

Yeah, it actually launches a new shell.

Yeah, exactly. Could you use it for a honeypot or something? Yeah, you could.

I've actually tried to do that, yeah. But it wasn't for a honeypot, but I tried it on my machine so if anybody compromised and logged in, script was running. It was kind of hackish, but if someone wasn't smart enough to actually know that, they wouldn't know.

Sorry, go ahead.

OK, so the program Script is a way you can sort of record a shell session and play it back later, which is pretty cool. So I saw it being used as an alternative to a screen recording. That takes up a lot less file size, and it's pretty cool. So there's a utility called Script, and you can see at the top it says, "Make a TypeScript of a terminal session." And TypeScript is what they call a file that will be generated of all the input and output that you create when you're working, and then it saves that as a file, and then you can play it back later. And then it also saves a time file which records the time between all your keystrokes so you can play it back and have it, you know, like how you typed it. So basically, what a better way than to just show you. So I think the way to do it on this version— most Linux distributions will come with GNU Script, which is easier to use version, in my opinion.

Do you want to use that instead?

I'm sorry. That's fine.

Oh, I've got it open right now.

OK. Just do it there. OK.

So now he's in my Linux favorite VM.

Yeah, OK. So you can see here the -t option will specify a timing file. On the old version of script, it outputs to standard error, so you have to just redirect that to a file. So if we do script -t, and we'll call that the file time, and specify the TypeScript file.

Let's call it TypeScript.

And so now it's starting to record.

So—

You can even open, like, a terminal editor or anything like that.

It'll still record everything.

And then when you're done, you just exit out of the shell, and it says script done, and the file is TypeScript. And you can see here we have a time file and a TypeScript file. And if we look at the TypeScript file, you can see there's a bunch of escape sequences which were from the terminal moving cursors and drawing stuff in different escape sequences. And then if we look at the time file— but you can see also in there is the actual content of what we were working on. You can see here's the hello and all that. And then if we look at the time file, These are, I guess these are second values and seconds of time between keystrokes. But basically this file, these two files can be read at the same time by a program called Script Replay. And that will take those two files as arguments and it will play it back just how you recorded it originally, so.

Pretty sweet.

And of course there's all this dead space because I was talking. And that's pretty much it. I think there's a few other options. I just found out about this today, though. I know there's an option for— it limits the time between— I guess this version doesn't have it, but at home when I was looking at the manual page, there is an option to suppress any long intervals. Like, for example, that time when I was talking for 6 seconds at a time, you could suppress that to a shorter time. And this one's pretty cool, the -d divisor. You can specify a multiplier for how fast how fast the recording will play.

So let me give you a demo of that.

So let's say we want to play it 3 times. And there's me typing faster than I can ever type. And yeah, that's pretty much it.

So a few things.

You don't have to actually type TypeScript at the end.

It'll use a TypeScript file by default.

Right.

So you can set your own file name if you want.

So if you want to type like, oh, Jon session, it'll write it to that. Yeah.

Um, another thing is it's really useful if you're making a new system and you mess something up. It's helpful to have a log of everything you type from the output of all of it. So in that case, a lot of times I'll just be in a new system, I'll type script, it'll have a log of everything I type after So I can look back if I screw something up or I want to know what I did prior to that.

Is there a way to like view a cleaned up version of the log file without all the tons of key presses?

You can use -less -r.

You still have the file there? No.

Oh, whoops, that's right.

Okay, hold on. If it's up to you, yeah, he's making one real quick. Want to? Okay.

-r, you said?

Yeah. Oh.

So that— go look at the man page for less and do -r because I can't remember. It has something to do with the terminal codes. It tries to interpret them or something.

Fancy, right, John?

Yeah, so it causes raw control characters to be displayed.

So you actually won't see them. Oh, and there's also—

I think if you cat it, it'll do it too. Yeah, cat by default does not show escape sequences without passing like -h.

Do a script with Vim.

Oh, you got the— yeah, do a script with Vim though.

And then Grunt cat it. I think it'll show something.

I can't remember though. So less -r. Or just— Yeah, just -r. OK. Oh, yeah, that's freaking out. And then do cat. Just cat that file too. Oh, OK. You're right. Oh, there was some that it didn't do. Yeah. If you modify the content of the file and save, can you still replay all the modified content?

Which one, the time file or the TypeScript file?

Oh, either one. Like, let's say someone just want to mess around with it.

Yeah, you should be able to do that.

So let's see. Of course, this is going to be garbly.

I'd imagine it'd work as long as you have the same number of keypress timing events as you had stuff in your script file.

Yeah. And I imagine it probably parses it from the top down. So even if the end doesn't work, it probably So like if I were to add something here, like—

and then maybe let's—

What's the second column, the keystrokes?

No. Let's try that.

Yeah, so that hello was something I just added, so that works.

You know what I want to do?

I never got around to it, but I want to write a program to use script.

Yeah, to actually— if you're two people on the same machine, connect the terminals. Instead of connecting the terminal, so you'll use script to actually replay it off.

I guess you can replay on their terminal if you have permissions to write to it.

Yeah, and you can actually use it like a training thing.

Oh yeah, and follow me, follow along.

Yeah, and you could, uh, you could couple that with screen or tmux or something and just put it like in the tab or something.

Yeah, you can do that too. I'll have to look at that later.

Yeah, cool stuff. Never knew about that.

And I'm guessing there is no limitation on time or capacity?

How much you want to drive?

Yeah, I would think it'd be file system disk, you know, one of the two.

Okay. I don't know. I've never had a huge script file though, so. Thank you. Okay, now we got, um, the VM vicinity up.

And this is where we talk about Vim. So just for those that have never been to LOG before, the idea here is you have these little sections to pack these full of all kinds of new information on all these tools so that if you stuck with LOG or you went to a lot of meetings, you probably know more about the command line than most Linux people you would meet if you just follow along with this stuff.

So that's—

it's kind of just introducing the new command line options. New ideas, etc., etc., for all these common programs. That's kind of how we do our sections. So hopefully you think it's information packed. And then we do have talks.

So I'm going a little slow, so let's speed some of this up.

Vim vicinity. So as we're talking about Vim, let's go to my configuration here.

There, let's do this.

So we have a few modes to do indentation. We talked about indentation in visual mode. Tonight we'll talk about indentation in mobile mode. We did that in a prior meeting on visual mode, which is essentially— so I want to do this, highlight a bunch of lines, and then I want to move them over, right?

Block, move them back.

So using the V key to enter visual mode, highlighting it with the arrow keys, or you would use J, K, L, semicolon, which is what I use, and you can move it around after you use the repeat command and stuff. So we'll talk about it in normal mode. So if you do greater than, greater than sign twice, it actually moves it over, and you can use the repeat key, the period, to move it over multiple times, and U to move it back. So it's a very simple way to do this. Also, you can do 3, so you can specify a number of how many lines you want to move.

So I want to do 5 lines, and then I'm pressing the greater than greater than sign twice. What mode are you in? This is normal mode.

So this is just when you're moving around, right?

So how are you— So hit Escape and you're in normal mode.

And then just do this, type this way, that the greater than, like, just like that twice. And you should see your line, the line you're on move, is indented. You see that?

Oh, cool.

So now do it with 4, type a number. So I'm going to do 4 and then do the same thing, the greater than, greater than, and hit period to move it multiple times. But this repeats the last command.

So that's that.

That's a cool one. And then we have This is moving a paragraph. Thank you.

Paragraph.

And I don't know if I'll have to match that.

I can't actually remember. I'm gonna go to the script that doesn't work here. Yeah, we need to get it. Oh wait, that's actually— let's see here.

Yep, so I did that, but it wasn't a paragraph.

So let's go to a shell script.

And it's got a bunch of lines you can do.

Uh, you can see that moved the paragraph right there, right?

So I'm using this command right now, this one right here.

And you can do that, that's to go forward and backwards.

You want to do the other direction though, you want to do the greater than sign again to go forward, right?

So greater than sign And then the right curly brace, and then I'll move a paragraph worth of information over. You can use repeat commands there too. So we'll do it here.

And so you can move it. Vim tries to figure out what is a paragraph and does it by the number of lines that are together, right? So you don't have to count every time. Then we have this one. And, Lucian, you know that off the top of your head? Is that just the—

I think that's—

Wait, hold on.

Ah, I'm an idiot. That's what I meant. So I'm using them.

I'm sorry. I mean, I got it now.

So this is the mod. Okay, so we're actually— help operators.

I think that's what I'm looking for. Where is the words going that you're typing?

Yeah, so we're actually— you can actually use a lot of these.

So in this case, we're using, um, we don't even see curly brace, curly brace. Where are you at? Oh, here it is.

So we're actually removing that.

Here we're going to use parentheses.

So What we're gonna do is on a function, it could be any programming language where it begins and ends with a curly brace, you can do that same thing and move the entire block of code for that entire function back and forth. And let's try to find one with a—

this one right here. This one's got the parentheses. Do the same thing here. Looks like that one looks— let's try. Here and then move the whole thing. So, uh, what happens if we did it on this? Let's do this, track that way, and we'll do— nope, that did multiple lines here.

So that one doesn't work on that though.

It's like, I don't know what the definition of that is. But so that's the quick ways that you can use it.

But I highly recommend visual mode is the best way though, because if I want to do, I can select it right here I just type B and then move around and move.

Real simple.

But the curly brace one is excellent if you just want to move this function. So one last time, I want to move this function, type 2 characters.

Ready, go. And now I have the whole function moved.

Alright, so next is indentation in insert mode. This is one that I've never used before until recently. So if you're actually typing, you're in insert mode. So I'm going to be in right now.

So I'm typing some text.

You can actually move that. Notice my cursor's at the end of the line, right?

And Ctrl+D to go back.

And actually, I've not even used this yet, but I'm going to try to. So that way you don't actually have to keep— you don't have to type if you mess something up or you're not where you want to be. You don't have to press Escape and then move it.

You can actually do it all in one, in the same mode. So in Insert mode, whoops, you just type your text and then Ctrl+T and it'll move it over.

Ctrl+D to go back.

So that's a really nice one.

That concludes Vim vicinity.

Oh, nope, I'm sorry, I got color schemes.

So last one, this is really nice, easy to do. So go into your command mode. So press the colon key, right? And type color scheme. And a lot of times you can go and tab it out.

Sorry, lowercase. There we go.

So now color—

God dang it. Sorry, my bad.

Alright, so there we go. Now color scheme, and now you can cycle through the number of colors for your terminal. So you can just hit Tab, and I want to change this to desert. Well, you can do Shift+Tab to go backwards, and then now it changes my Vim color scheme to the desert color.

And again, you can just Tab through.

If you want to match on something, you want to start with a T, just match on that, tortie.

Look at that.

Looks all right, not bad.

Not a bad color scheme.

And then you can go back. I'll go back to my default, which is Solarize.

So again, color scheme, and then just tab to the ones you want. So that's the way to do it. Devin, we'll do—

you want to do it last?

Or, well—

It's actually really short, so—

You want to be our dessert?

Your machine? Yeah. You have no dessert. What the heck? I win. What?

Creepy mofo.

It would be creepy if I said it to one of these guys who I don't really know.

But Devin— or if you want to get on the Zoom call. That's what I'm saying. You have a client, you can just, uh, I can just— It's not free software. No.

That way we have, we can actually refer back to the source.

That's right.

Balin, do you ever look at the program VRMS? Like, uh, it stands for Virtual RMS.

Induction on the non-free packages.

Installed on your system.

Does it really?

Yeah. That's awesome. And it's—

how do I respond?

VRMS. OK. Virtual Retrieval System. So it does. Installing.

And presumably someone has a cow file for—

Can we get Mad Dog to come talk?

What? Leland.

Probably get Mad Dog to come talk. If he doesn't die by then.

Remember Mad Dog?

He consulted on the Matrix movie. What? You know he consulted on the Matrix movie?

He did?

Mad Dog. He's the one that put in the bit about Trinity exploiting the SSHD1 vulnerability to bring down the power grid and save Neo. He wrote that for the movie. Nice. At least that's feasible. Yeah.

As opposed to, I'm going to write a GUI interface using Visual Basic.

Or an IP address.

See if I can track an IP address.

Or Emacs. It's not free. All right.

So today I'm doing bitsets in C, which are actually a really neat way to do certain calculations with sets and take advantage of hardware-level parallelism because they all involve bitwise operations, which are extremely fast. So the basic idea behind this is, let's— you have some unit that's, of course, only useful for finite calculations. You have some universal set of objects. And you want to do operations on subsets of it.

You want to take use of the things You want to take intersections.

You want to see whether one thing is a subset of another. You want to see whether certain objects are in certain sets, you know, or whatever. So the basic idea is you have your universal set. You impose some arbitrary ordering on it. And then, so what the zeroth element, for example, and then you— so you have your words, for example, 32-bit integer. Ints on 32-bit machines, 64-bit ints on 64-bit machines, whatever. And in principle, you can extend this to a pretty large number of elements by just having an array of words instead of a single one. But, you know, for this, just to keep it simple, I'm only doing one word. So depending on the size of your machine, you could have universal set of 60 with 32 or 64 elements. But, yeah, so the idea is in each position, in each bit. So each, each word is considered as a set, and the 0— the 0 in a certain position means that that particular element is not a member of that set, and a 1 means that it is a member. So right away you can see that there are 2^n possible different subsets because there are 2^n different binary strings of length n.. And then we can do all sorts of operations on them. So to create a new bitset, all we have to do is just return 0 because all the bits are 0. To take the complement, it's just the bitwise complement. And then to get the cardinality, we have to calculate the Hamming weight of the string, which is the number of 1s in it. And there are all sorts of algorithms to do this quickly. The naive way of testing each bit and adding up the cases for it, whereas one is really slow because of all the branching and stuff. And there are plenty of branchless algorithms to do this, which we could do, but that would take forever.

So I kind of cheated.

GCC, maybe other compilers, have a built-in that actually converts this in some cases to a single machine instruction. Some CPUs are capable of computing the Hamming weight in a single instruction. So whatever is best for your particular machine. It'll do that. So you can just add up all of the ones and get the cardinality of the set right away. To take the union, it's just the bitwise OR. Take the intersection, it's the bitwise AND. To subtract one set from another, you just— you are taking the union with the complement of the second set. So you're saying all of the elements that are in A but that are not in Be B. to take the symmetric difference of two sets, that is, all elements that are in one set but not both sets, which is the exclusive or. And you can toggle whether you can remove or add individual objects to a set. But if you want to add them, you just take the or with one shifted to the correct position. To remove them, you take the bitwise and with the complement of 1. So this is all 1s except for a 0 in the particular position when you take a bitwise and. So that'll set the bit to 0. To check whether A is a subset of B, you, uh, just— so this, right, this is a logical implication. So this is saying A implies B, or in other words, for each bit membership in A implies membership in B. So in order for A to be a subset of B, all of the bits have to be 1, and which is the complement of 0. To test whether it's a member, you are— all you have to do is just take the bitwise AND with the 1 in the appropriate position. And I was inspired I was inspired to do this because I was talking to Shane about one of his homework problems, and this came up. I mean, we did the calculation mathematically, but I also wanted to write a program. It was involving a graph where the vertices were sets, and we wanted to count how many edges were in the graph. And the edge set followed certain rules.

So I took—

so there were 64 different vertices. So for each vertex, I compared it to all of the other vertices. And for each comparison, I did a subset calculation. And if it satisfied certain properties, I added 1 to the number of edges. And when you sum it up over all the possible combinations, get the final answer. And there's only a few lines of code for a very complex calculation.

So yeah. This is just—

and I mean, this is a very nice method of doing this because obviously CPUs are designed to do all the calculations on the different bits in parallel.

So you get extremely fast computations of these basic operations, so it's useful for all sorts of things.

And yeah, that's it. Any questions? Does anyone know what's going on?

No, I don't anyway.

What's going on?

So you have—

I—

you guys don't know what a set is, right?

Sure. This looks like a set, like a full map.

Yeah, like a collection of objects. In this case, it doesn't really matter what they are.

So you have like—

yeah, so it's just like a bunch of objects that you put in a box, and the subset is you take out maybe some of the objects but not all of them. It could be all of them, you could take out none. You just take out some objects and that's the subset. And a union is when you mix two things together. An intersection is when you see what elements they have in common and all that stuff.

So I just thought—

sorry, Devin, I know what you're talking about.

I just—

when you asked that, I wasn't paying attention.

Yeah, rude bastard. Okay, that's it. I'm a member. Oh, I wish I understood it, but that sounds cool.

Step forward, Prophet.

Did I miss that part? Is there a restroom on this floor?

Yeah, it's right there, the other side of the elevator. But I have a question. You use that built-in, right, PopCap?

Yeah.

Like, how does—

how is the processor able to do that?

Do you know?

In one machine executive or execution, whatever.

However, however they design it, I mean, it's— I don't know how that works on the circuit level.

Yeah, because like if they can do it with one machine instruction, can we emulate that in the software? Probably in one execution so that So that regardless of the architecture, we can do that.

I mean, there are very fast algorithms to do it. But I think doing it in one cycle would require specialized hardware.

OK.

But there are branchless and loopless algorithms to calculate that so that you don't mess up the pipeline.

So how did you—

your problem was a graph and you had to show it up here.

How was that graph presented to you? How did you get it?

You had the set S containing 1, 2, and 3. And the vertices were power set of S cross power set of S. So you had 64 different ones, right?

First— just by our assumption. Yeah.

And the rule was there's a directed edge between A and B if the first component of A is a subset of the first component of B.

OK.

And the question was to count the number of edges.

Got it.

We did that mathematically, and then I also wrote a program to use these operations.

Are sets in regular, like, NCC?

Or is it—

in what? Were you showing C or C++? Pretty similar. I mean, you could do the same thing in C++. Yeah. Separation. Mhm. Let's just say, like, hit me random things. All right. Okay, on to the small talks and then we'll be done.

Okay, so the first talk is just more Docker stuff. I gave a Docker talk a while back. Um, we're just going to talk about just some small— a few, like a handful, or even a handful, like 3 things I thought pool, and if I can remember them.

I just read my notes down. So one is—

anybody use docker exec? That's a new Docker command.

Okay, so we'll talk about that. That's an easy one to talk about then.

So that was released in 1.3 or 1— I think it was 1.3 or 1.4. So we're on a host. This is a little out of systems. And let's see here. We have docker help and we can see that there is a new command called exec. So it basically takes an existing container and allows you to attach to it and with the intention of running some sort of command. So we can actually do—

so we have— let's check the containers right here.

So I have 3 containers right here. We have cAdvisor, Ubuntu, and and we have islet. So we can go to any one of these. Let's go to the islet container. So we actually need the container ID in this case. It works on containers, not images.

So we're going to docker exec and we'll do the help for the options.

You can see there's a few options here. Interactive, um, tty.

We want—

if we want to have a shell and enter a shell in that container, uh, we would actually need to have a tty allocated so we can do stuff. So, we'll do docker -it for interactive, true, the container ID, and then what we'll do is run the command. So I can do ls, but that's really not interactive.

So I ran that. top. I wonder if you can do this. What if you can term equals vcluck100, for example. Actually, you can do it.

Well, you can't pass that in on Docker exec, but you can't really use it with a regular command. Anyway, so we're just going to do bash though. So this is how you fix stuff. If there's a running container, say you have like in our case IIS server, and you need to fix something while it's running in a particular container, like if someone, a user is in there doing something, you can attach to it in this particular way and work on it, find out what they're doing, make changes, etc. Or if you have a web application that's running in a container and you want to make a slight change, you want to get some logs off of it, etc., you don't have a better way to do it, You just run docker exec interactive terminal and then say which container ID to attach to and then give it the command you want to run. So I'm bash and actually in this container while it's running. So, okay, we don't have—

let's try to fix that. Export term equals VT100, which is an old terminal, and see if that does the trick. I guess it does.

So now we can actually run top and view stuff in the container live while it's— so there's thousands of web requests, HTTP requests, this machine, for example. I could actually run this and see how it goes inside my container and see what it's getting as far as data, processes, etc., etc.

So that's just a small one. That's a fairly new Docker command that's very useful.

What else we got is, so in the case where— anybody know what a bind mount is in Docker?

A what?

A bind mount. It's essentially a mount. They call it a bind mount. But yeah, mounting something from the host to the container. So we can do bind mounts for not only directory, but you can do bind mounts for particular programs. Anything on the file system, really. You can actually put /dev/null from the host inside the container. You can put your MySQL Unix socket from the host inside the container. You can do your X11 socket from the host inside the container. That way you can actually use those commands inside the container and directly talk to the host. So in our case, I use a bind mount to mount Docker inside a Docker container. So inside a Docker container, you can control Docker on the host. So it's a Docker in your Docker kind of thing. And you can actually see here—

oh yeah, I don't— you know, there's a terminal right here.

We'll go to this directory. I'll show you the commands I ran to actually do this.

So, uh, uh, dash, there it is. Okay. So, We're actually— oh, that's what it— that one does it too.

Okay, so here's mine. I'm running a docker run. My name of the container is islet. And here I'm specifying the bind maps. This particular one is taking the /usr/bin/dockerclient from the host operating system, mounting it in the container so that when you run Docker inside the container, it's actually communicating with Docker on the host. So you can be in a container and control Docker. And what you don't have to do is— 'cause your Docker images and containers are stored in /var/lib/docker/mount/storage-backend-aufs or device mapper, for instance, and then the directory structures are down there. So to actually utilize that to create containers from within a container, you actually have to mount as well that particular file directory structure as read/write inside your container. That's exactly what I do.

What?

Yeah. That's true. Yeah, that's pretty awesome.

So here I mount /var/log.

Yeah, so you can—

wow, okay, that's good to know. You can like commit the container you're in from inside the container.

Yeah, you could do that. I don't know if it'll let you do that in a running container, but—

Yeah, no, you can.

Oh yeah, that's right, that's right, yeah.

Yeah, you commit running containers.

That's right.

I didn't even think about that. Yeah, that'd be pretty sweet. I don't know, it's fucking cool. So also is, if you want to do firewall rules from inside the container, say you have a container that's a firewall, right? And it's intended to have some scripts to do some firewall management. You mount iptables from the host system, you can then use that container to actually manage the firewall from the host system. So if it gets a packet, it can block it or something, apply the appropriate rule. So that's actually really cool. But you would mount— like in this case, I mounted as read-only so that nobody can modify that. Because if you modify in the container, somebody could also modify it on the host if you can write that binary because it's the same one, right?

It's just mounted.

So you always want to do those read-only. I also mount some of the stuff from the host. Other examples are—

sorry?

But when you're running it, you're running it with the privileges of the host, not the privileges of your container?

No, it's actually the privileges of the container.

So you have to make sure that your container is configured to have the appropriate privileges to do it on the host.

So in the case of Docker, You can do that. You just have to be root inside the host or add your user to the Docker group inside the—

I'm sorry, root inside the container or add the user to the Docker group inside the container and that will allow you to do it.

And there's a difference depending on the type of stuff.

If you have a—

there's privileged containers and unprivileged containers. Depending on the software that you want to run and how it's exposed, you may have to use --privileged with the Docker daemon. I haven't had to do that, but for the most cases it seems that iptables, which is a root tool that you have to use, become root to actually apply rules, and same with the Docker binary.

Besides the fact that you can be part of the Docker group, you don't actually need to do that.

I'm surprised because doesn't iptables write stuff into the kernel?

Yeah, so as long as you're root inside the container in this case— well, actually I shouldn't say that because I actually haven't tested that. I'm just guessing that it works. I probably should have tested that before I committed this anyway, but I did, so we'll see. 'Cause this is a feature of this program I wrote that's not very— I don't even know why this used it, so I need to do more testing on it. But, so it's there.

I haven't tried it, but yeah.

So what we can do is—

Is this a hack from using Vagrant to do this?

We can actually try it right now, though.

Huh?

You're saying, like, this is a Vagrant hack on Docker?

This is not— No. A feature of Docker. Huh?

This is not a feature of Docker that you—

Oh, you mean the privilege stuff?

Well, no, like just being able to, like, manage Docker from inside of a container.

Yeah, it's not—

it allows you to do it, but it's like, it's more like an abusive thing. I don't think anybody's doing that.

It's a Vagrant hack, right?

No, it has nothing to do with Vagrant, but I would say it's a Docker hack, or on an RTOS hack, you know what I mean?

Yeah, yeah, yeah. Oh yeah, because this is just a bit—

this is just bigger and happens to be using those Mouse.

Yeah. Okay.

Well, in this particular case, I'm not using Vagrant at all.

This is just an EC2 VM. Oh, but I— you were in a Vagrant file.

Oh, that was a different window. This one's actually an EC2. Well, yeah, in either case, it's just a VM, right?

So it has— that's—

there's nothing to do with that. But let's try it though. You asked about the iptables. Let's go to this and we'll do— docker exec, right? We'll use that command we talked about prior.

Okay. And—

oops, that's not what I wanted.

Dammit.

Alright, so I'm in there. Now what we're gonna do is we're gonna run iptables. Ah, looks like I can't even run that. I need to mount the libraries from the host in that particular case. I probably should check that out.

Let's try sysctl.

Sysctl seems to have something, and this actually modifies the kernel as well. So in this particular case, depending on the option, we will set something.

Let's do a—

let's change, I don't know, dirty bytes, memory paging shit. So this will have to make a write to the kernel. So let's do sysctl -w=1, and this looks like I cannot do that for this particular case. Will that happen for every one?

And why does it say revoking? Setting key— let's try that from the host first, actually, because I've never set this value, just to make sure it works.

Yep, it does there, so Let's go back to the container and let's try a different one.

I think it's going to do that for everyone. Let's set the loopback MTU to—

this is going to be required to be root too.

Let's set that to the MTU of an Ethernet frame equals 1514.

Yeah, so it looks like that doesn't actually work either. So what we'll need to do, I think, is we'll need to run it as a privileged container.

And then it should work.

So this is right here, is by default privileges false for your containers, and if you do is true, that gives your containers extended privileges. So that will allow you to, I assume, allow you to modify the host in that particular way. All right, but I can, I can, but in that container, as a normal standard user, I can in fact work with Docker from the host.

I've actually tested it.

It works. Like, every time I log in with that, I use Docker, like, the run. I can create a new host on the actual host— or in the container on the host operator.

So, you could tell Docker to give you elevated privileges?

Yeah, yeah.

Okay.

When you actually create the container. If we're talking about elevated privileges, we're talking in terms of the container. So, every container is default To have a limited set of permissions?

I'm saying if you have Docker inside your container, can you real-time reset the privileges of that container so that you can then do other stuff?

Good question. I don't know. Let's find out.

So docker run --privilege=true it bash.

The image I want to do it with.

Ah, looks like I have—

I did run a privileged container and accepted it inside from the container.

So a container with a container at this particular point.

Now I don't know if it—

since it's ran from a container, I don't know if we'll actually have the ability— we don't even have IP tables on this one. This is control. Let's try that, uh, what I did earlier.

It's just Ctrl+A. Or let's just do that. And then—

and this is probably going to give you the same thing, I imagine.

Yeah. Oh no. Yeah. Well, hold on.

Yeah, I just modified the, uh— well, no, I mean, I'm in the container. In this particular case, I did not modify this because I didn't bind mount the, the, um, the sysctl, system control tool from the host.

So I actually need to do that.

So we'll need to do -v /sbin. It's getting complicated.

Sbin syscontrol.

Okay, so now that should be the syscontrol from the host.

And from there— but you're proving the opposite though, I think.

No, syscontrol -w -i 0. Ah, let's change it to 1 and let's check the host operating system's kernel. All right, so now, um, let's go ahead and open a new one up.

There you Alright, and sudo -i sysctl -a and we want to grep for commit. Yeah!

So, okay, so here's what I'm in a container, launched another container from within that container with the -s privilege option, and by now the sbin sysctl from the host operating system I'm 2 containers deep. In that second container, I set sysctl values for the memory paging system in there and it did in fact modify it on the host container kernel.

This is so awesome.

Alright, so yeah, you just need to pass the --privileged option. Shane, you missed all that.

That was awesome. You're obviously being a rude douche today. Who is? Me.

Anything reported?

I'll put up the mic. So next thing.

So the other thing I want to talk about real quick was the file structure in Docker.

So let's do— actually, let's go— yeah.

Okay, so let's put it over there. So if you want to find out what your backend is in Docker repositories, it is this. It's the repository's dash, the actual storage backend driver. In this case, I'm using AUFS. You can do docker info and you can see that the storage driver is in fact AUFS right here. So this is how you're able to detect on the fly by doing a file glob with the shell and getting the data in there to find out what storage backend you're using if you're not using the Docker info command. And you don't have to spawn that process. You don't need to spawn Docker and then run the info command, which is extra, especially if there's a blocking or locking issue, which I've had before.

There's a lot of containers running.

It can be really slow just to get the output Docker command. So you can actually just read from the file system and just read from this repositories-whatever the backend is. So if it's the backend device mapper, it'd be /var/lib/repos/deviceMapper. In this case, I'm using AUFS. You can see I have a JSON image or a JSON file here that has all kinds of information about your current Docker setup. So here, this is actually a file that refers to every image in Docker that you have. So you can see that I have my Google Sea Advisor one, my islet one right here, and a few other ones. This is how you're able to query the image using the JSON tool and get the image ID or get the image name for your particular Docker installation.

Where is that stored?

Say again?

Where is that stored?

/var/lib/docker/repositories. It's just a file, but the last part Is you have to specify that the file name is repositories- and there's storage backend driver. So if you have Docker installed now, you're probably using AUFS or you're probably using device mapper.

Yeah, device mapper.

So then type device mapper at the end of the cat file and you have that JSON data. The other thing that's interesting with Docker is—

what else is there?

There's JSON data for all of the containers and you can actually pull out information, the actual settings of the container. Of course there's a library, you can import Docker in Python, for example, and query it that way. But you can do it from the shell too because of all these files. So this is the last example I'm going to give you for the Docker section. So we do Docker or bin or cron.

I'm just looking at my scripts that I wrote because I don't remember all this by heart. Which one do I use? Is it this one?

Yeah, so /var/lib/docker/containers/config.json is the file. So you go to /var/lib/docker/containers and there's the containers. Right here's the container ID, the hash, all the way down and you can see there's entire file systems or mess with the file system, that's actually container information. And you can see there's a JSON file, so let's go to one of the containers.

Let's go to any CC.

And then we go to—

here are the files there.

We can do config.json. This is the JSON object of all the options that are passed to create that container.

So you can actually use this to see, oh, this guy—

and I like the shell script. I just do fgrep and figure out if this particular setting is set. And then whether that is, I run some other code. So that's how it's actually— how it's all recorded.

So this actually determines how the—

this shows you how the container was built, which is really nice.

Can you explain fgrep? Basically just— fgrep? Yeah.

Yeah, so fgrep is the grep tool. It stands for fast grep, and it's just a way that it doesn't actually use the extra extended special character matching, the regex.

It's kind of like the opposite of -ge.

Yeah, yeah, actually it might even be—

let's test it.

So you're saying fgrep is grep without regular expressions?

Yes. But I did say extended solution and I actually now that I'm thinking about it, I'm not sure if it's extended or if it's just all regular.

Just by default it accepts regex but not extended regex.

Can we just use grep? Say again?

By default grep accepts some regex but not extended.

That is exactly correct.

That's correct.

And that's what I'm saying, I don't know whether it's the default standard regex and the other one's called the extended regex.

Is that grep -e?

Cut?

What is grep?

Uh, grep -E is the same as egrep and that is what gets you the extended regular expression set. But we're talking about fgrep and fgrep does not allow you to use any of the extended, well, this is what I was talking about. We don't know whether it nullifies or avoids the extended regular expression set or the entire or the basic regular expression set.

There's actually two sets.

That's what -E is.

It gives you the extended set. So why would you use fgrep if you could just— Because it's faster. Because— well, hold on. I'm not finished asking my question. I guess what I was going to ask is, couldn't you just not specify a regular expression using grep? Start with an ak?

Yeah.

Is it that much faster?

Well, if you wouldn't have to escape any regex characters.

Yep.

OK. Yeah, fgrep just interprets the pattern as a literal string, the same as grep -a.

Okay, so it doesn't even do the basic set. So it's just a faster way of doing it. So I always recommend like in shell scripts where you know the exact pattern just to use that. It's a quicker way to do that, especially large files. And we can actually, if we have some too large files, you can actually test it side by side and see what the time is. Yeah, for each one. No, actually I might do that in graphic for the next meeting. Um, but yeah, so you get the same man page. So look, if you just go to grep, you go to grep man page fgrep, you can see as— what's your name?

Sam. Sam.

As Sam mentioned, it gives you a little information. You can see that it—

oh, hold on here. Yeah, he says same as grep -f.

If you look for -f, let's match that, and it says fixed string.

So just as he said, it's a literal.

It matches literally. So in this case where I have a file with comments, this should not work.

I want to match the hash.

What's a good file?

Anything, honestly. Should have comments. .com. Should have comments, yeah.

And you can see that that returned 1. And just to verify—

Oh my god. I'm like, again, sorry. Because it's going to make our substitution to B easier.

Yeah, there's comments in there, so it didn't match.

That's right. Yep.

So you see that, Will? It's just a file that has comments in the first line, so it's matching the first line.

And it does not work.

And if I do grep, though, it finds the comments.

Because it's matching a literal caret, that you could do something like that.

Oh, I see. And the hash.

That would be convenient for searching through code. When you want to search for things that would otherwise totally mess with your regexes.

Right, and you wouldn't have to escape them.

Right, like asterisk or period that match a single or multiple characters. So that's when you would use empty quotes. Right. We put that in the notes?

Like that command you just showed?

Yeah. You've got to switch the order of everything. It's literally a caret and then whatever character you want to match in the file.

Uh, so what you're wanting to match the all lines that begin with the comment character?

Yeah, yeah, that's how you do it.

That's what you want in the notes.

These two lines. Yeah. Oh yeah, I can remember that. So okay, quantities, we can substitute them. And if you want to match something at the end, you just do the dollar sign like this. I know that'll be p5a.

That'll match all the colons at the end of the line.

Oops, gotta escape, uh, Oops. There it is.

Okay, so once I put the strings around it, that's right from the shell.

Okay, so that's how you measure it at the end.

Okay, so we're done with Docker talk. Quiz, this will be real short. So I was reading over Christmas break, I was reading a lot of books on neuroscience and memory and stuff, and what I've learned from I did some research on some really good books, and one was by Nobel Prize-winning economist and cognitive scientist Daniel Kahneman. He wrote a book called "Thinking, Fast Thinking, Slow," and he talked a lot about a lot of different things, and one of the key chapters focused on memory. And there was another one called "Make It Stick: The Science of Successful Learning," which is by 3 well-known cognitive scientists. You can find the book here.

It's got really high reviews.

And they talk about memory and how to improve your cognitive performance to remembering and stuff. So essentially the 3 takeaways were you need to incorporate in your learning— don't do things like rereading things multiple times.

What you need to do is practice what's called retrieval practice. It's essentially the thing—

this means practice questions, have a quiz. You also need to incorporate a thing called interval and spacing, and they refer to interval as spacing time out between the quiz. And the idea is the amount of effort that it takes to remember something is what actually opens up new neural pathways so that you can actually recall it quicker. That's one of the things. So in the case of intervals, if I tell you something, it's easy to remember tomorrow, right? If you do a quiz on it tomorrow, it won't help that much. But if you do a quiz on it a week later, a month later, and keep spacing out more time, you'll actually be better at remembering it. So you constantly test yourself. And the spacing was conceptually space out the questions around similar material that is not exactly the same but is related. So particularly if we're talking about Docker, maybe it's particular commands of Docker and the particular options for each particular Docker option or arguments for each Docker option just to get the whole thing or talk about the concept of containers, et cetera. So I wrote this— well, me, I started the shell version of this program and my friend started doing PHP. Please, somebody write it in something better if you have time. I'm sure it can be written in C very quickly or Python. But essentially it's just a command line program called quiz where you just feed it in a file, a text file, and which is basically a CSV format of questions and answers. And you specify a delimiter if you want that's not CSV. And it just reads the questions to you. So we got, like, I have a bunch of them. So my idea is for me to actually improve my learning, every single book that I'm going to read, I'm going to create a new folder. I just use MakeDir, you know. And then create and get the list of things that I find I learned new in a book. Write a question out about that particular thing and then possible options or answers. And then the format is, as you can see in the README, is the question, the question mark, and then CSV. The first field is the actual correct answer and the other ones after are the other possible answers that you would have in multiple choice. So when the program parses it, it knows that it stores this one in the answer variable..

And for these, these are just randomized to the user.

And it's just a quick way to create a quiz program that you can actually teach yourself things. So in the case of PDFs, you can actually, strings or use some other program to parse PDFs and pull out all the questions using regex and then use that as a basis if you can format it properly for this.

So you can actually take textbooks or PDFs and make quizzes out of this. We did a quick demo.

Actually, I don't know how many changes have been made since it was last done.

So let's skip forward.

It seems like a fun idea, but how do you grab text from a PDF in such a way that's not completely garbled in one piece?

You'd have to use a program that will just match the ASCII. There are programs that allow you to parse it. For example, those PDF to HTML tools and stuff like that, or PDF to text tools. So you have to use one of those.

I know, oftentimes they come out with random spaces in the middle of words. Random character substituted.

That's right. It's not going to be easy, but if you can do that, use maybe some regex to trim off that stuff, presuming there's some pattern, you know what I mean?

I'm not going to say it's easy, but you can do it.

Well, I've been doing things by hand. In this particular case, we're going to do -f, and then I wrote— I read a book on sudo recently, and so I can specify this sudo.csv, and here's the questions.

It's a very simple quiz program, right? So you can be like, "Oh, 1, correct," and move on to the next one.

It's just a simple way to do it. I'd like to add a lot more features.

But that's Quiz.

So if anybody wants to take this and rewrite it, I haven't actually had time to do anything with it. I've been so busy with my other projects. But I have the shell, the start of the shell script.

The iffy part with the shell script was sorting it.

The quiz is going to have different sorting algorithms. You can do sequential or randomization or something. But doing that in shell is a little bit tricky. Unless you want to use a sort program which kind of gets uglier. It's a hackish way to do it but it works.

So there's that. So we're done with quiz.

But if anybody wants to add to this, so Linux stuff, if you're working on anything Linux, find an address that you can pull my repo and just send a request and actually have that file full of your questions and answers. And this is proven learning techniques. And the last talk is—

That was appreciable. Pseudo in depth.

We're actually probably not going to go into depth. Um, so here, who, who knows?

Who's familiar with pseudo? What?

Who's familiar with pseudo?

Not in depth, at a basic level. Okay, so fix me a sandwich. Huh?

Fix me a sandwich, right?

So, um, make sure you gather a root of script that would interact with, uh, Jimmy John's website.

Yeah, but it didn't use a Makefile, which I think he missed a big joke by skipping that.

Thanks.

So sudo is a program that allows you to run commands as another user.

Don't get silly, Alias.

The default is typically root, but doesn't have to be root. If you have a web server and you want to give a group permissions or user permissions to write to that web server, and you want to have another user be able to become that user just temporarily for a particular command, you can do sudo -u that username and then the command they want them to run, for example. So it's a way to provide access control to different users and it's very good. It's better than just giving them the other user's account. Like if you want to give somebody root, you don't want to give them the entire shell. With sudo, you can limit the amount of commands that they can run. And for admins, it's an admin stream tool.

But do you need somebody's password to be able to do that?

Do that? Like for me to run as somebody or because I'm sudo I can run it? What'd you say? Like do I need his—

if I want to run, you know, as dev, do I need his password?

Right. So that's a good point. The thing about sudo is that you only use your password, right? That's the advantage over su, which stands for switch user. They use the password of the command—

of the user that you want to become in that particular case.

Like su - right? So I have to know the root password to become root. In this case, the dash says, hey, we're going to become root as the default user.

You don't need a dash there, do you?

You don't have to.

It's an environment variable, right?

Oh, it might be, actually. I think you're right. So the dash is an environment variable?

No, it's about preserving the environment or something.

Yeah, you're right.

Because I usually just do su.

If I need to actually do su root, there's a dash. And then— That's right. Oh, here it is. If define the command— wait, hold on. This— I wish there was an easier way to find it here.

Anyways, I think that one's right. That's my bad.

So let's look for default.

OK, the default is becoming supervisor.

So su would become supervisor. That's right. So a common way to use su, I don't have, I don't have, well, I can use it on the VM. It'll work, I guess.

Is, okay, so this will work. su became root.

Oh, that was, there we go.

Okay, so I'm root now. So I had to authenticate with root's password in this particular case. But if you want to do something else, -c, run a command as that particular user. I want to run cat /etc/shadow, right? The shadow file has all the user hashes in it. If you're using a local authentication, then you can do this. I have to know root's password, which is almost pointless in the case that we need to actually use it to prevent people from becoming root.

I'm sorry, I typed that wrong.

In this case, I actually cat shadow because I became root just for that command using the -c.

And it actually launches a non-interactive shell.

So the difference between that and sudo is I don't need no root password. The root, the administrator of the system can keep the password secret.

He doesn't have to tell anybody. You can just delegate things to different particular users. How do you do that?

Well, the beauty of the design of sudo is that there's one, it's designed to have one simple configuration file that can be copied through an admin in an organization that has hundreds of thousands of systems you can have that same sudoers file copied to all of the systems across the board, and as long as you account for every condition for every user on any system, it actually will work.

You don't have to maintain different files for every system, so it's excellent in that regard.

I don't know why I opened this as HD, but I did.

I wanted sudoers.

So first of all, important, use visudo. I've learned the hard way in the past for this. This actually checks so that when you actually exit, visudo will open up the sudoers file inside of an editor that when it closes out, in this case you can set it from the editor environment variable, in my case it will be vim. When you close out of it, it will actually do syntax checks. If it finds out it is not valid, then it will let you know and ask you to re-edit the file because if you exit and it's not valid, you could prevent yourself from logging into the system ever again. No. So in this case, it looks like it's using nano, so I need to export my editor variable to nano.

Hey, I want to use Vim because I like, you know. Okay, so now I have this open.

And defaults, just a number of defaults that you can specify as a secure path. This is a path to where anything that any program that sudo is going to use needs to be located in. And it will not run programs that are not in that path.

So it's called the secure path variable.

And the cool thing about this is the syntax, the language that it uses to actually configuring. configure So by default, it says that this line is for the root user, and the next line says that this is going to match on all hosts. So if you have a system where you have multiple— or you have a configuration on multiple users, this actually can use it on different hosts. So I can say, well, I only want to match this on the web server. So only on the web server, if the hostname resolves to web, then run the commands as any user that you want and you can use any command. So the idea here is user, host, commands that you can run as— or users that you can run as or groups that you can run as and then the command. So let's just go to a better example that we actually use on Hanyu.

Oh, that was Hanyu.

Wait, hold on.

Let me go to a different one. So, by the way, real quick.

Sudo -c is how you actually check the sudoers files to verify their syntax. If you ever make a change from a shell script, always run this and check the exit code of that. But let's see, sudoers.d/log. So on our dev server for log, which we have, we have— so if anyone wants access to it, you can have it. We have a ZenProject server you can spin up your own VMs if you want to do your own development or stuff. Just let me know. So in this particular case, we created a host alias called dev_systems. That's what VM on this is called, dev. So we can assign multiple ones if we want. It can be web, Docker. So you can have multiple systems. It's just whatever the hostname command resolves to. And we can say that wqh, which happens to be Devin, is allowed to run all commands without a password as root on only the dev system. He cannot touch any other system. And that is because we get— that is Devin's system. That's his development system. I give him full privileges. So Devin is able to do all these things. Now, this is how you lock it down. So this execute file is copied to each virtual machine as they're created. Another example would be If you want to have Devin run commands as Jon—

whoops. You just put that in there.

That tells— that says when Devin runs sudo, he can use the -u option to actually run as Jon, as me, for example. It doesn't have to be root. If you use -all, then he can run as any user. He can run as any user. You can do further limitations. You can do So if you wanted to, Devin is only allowed to run Docker, right?

Right. He's the only one who can do that.

Then you can do that. That way, anytime the dev— that's the only program that Devin can run as root, right?

In this particular case. So you can specify commands.

You can do it like this, Docker. So we'll give Devin ls as well, right? So you can just put these on there and build it.

Chris came early.

So you can actually do a command list like this. So it's like cmd and then you can actually set a bunch of commands here. So we're going to say Devon's commands and then you can actually set that with all those ones we just did. And so in this case, Devon's allowed to run /usr/bin/docker. He's also allowed /usr/bin/ls and then we can just devon's commands right here and then sudo will take care of the substitution. So that's the power of sudo. There's a lot of cool things. One that I recently learned is that you can actually do hashing. sudo has an option to take the hash of a program. So I can do md5sum of say /bin/ls, get the hash, stick it in the sudoers file as the program at the command part and you tell what command it is and anytime a user tries to run that command, if the hash differs from what you specified in the actual file, it will not allow that user to execute the command. So you can actually prevent people from maybe if they're gonna Trojan an sshd binary, right? So you can actually say, well, you can't do shit with this unless the hash matches. We know that it's the same way it was when we created this configuration.

That's never happened on a production system before or anything like that, especially not on campus.

SSH decharging? Yeah.

Yeah.

So this would help in that case.

Especially not on a campus cluster.

Now one problem with this is you have to update the hash frequently, especially if you apply updates, binary changes, but it just depends. Maybe you can automate it in some way. The other thing is, if you come up— here's from an attacker's perspective and useful for the user as well. But if I were to go on a system and I know it had sudo installed, the first thing I would do is run this command, sudo -l. That actually tells you what your permission, what you're capable of doing. You can see that I can run all commands as root, as all users actually. So it tells you which host I can do that on too. So that is, I recommend learning this tool. A lot of bad hackers, you'll see in honeypots, They'll just run sudo or try a bunch of stuff and it'll fail, try some more stuff. When literally all they have to do is sudo -l, they can find out what they're allowed to do.

So, saves you time too.

Back when I was doing ACM admin, I know what most people would do as soon as they got an account on the ACM clusters, they'd do a sudo ls to see if they had access to sudo. And occasionally you'd get something really funny like sudo make admin get email. Yeah, it was really interesting case study.

Nice, nice.

Yeah, so yeah, here it is. So in a case where it was allowed to run those particular commands and they wouldn't know it, say they log into an account where you have the, you know, we list the specific commands like in Devon's case, /usr/bin/docker and /usr/bin/ls. They try all these other commands except those two commands maybe, right? And then those would, all the other ones would fail, but they didn't know that they actually had permissions to these two commands, that they can potentially abuse the system.

It just depends. But yeah, it's pretty useful.

So you can check individual files, -f, if you just want to verify. I want to do sudoers -d. I want to verify the log file.

I'm gonna do -c and that just— oh, it becomes sudo for that.

And then check parsing is correct. If I mess it up, it'll let me know.

You don't want to do that though. Jon Schipp. Sure, I'm trying to think.

I'm just doing this off the top of my head.

I'm trying to think of, is there anything else that was particularly important for the turn and hold?

Do those limitations and how it's not quite the same as SUE into root?

Yeah, so we talked about how I think the big advantage is that you can delegate particular commands on particular hosts for particular users without knowing the password or how that user authenticates, that particular user that you want to become. In this particular case, sudo just uses your level of authentication.

It will just pass with your password.

So that's the big difference between sudo and su.

How do you block sudo -tagadd? If you pass the -s flag to sudo, it will give you a shell.

That's correct.

How do you block that?

Let's try— what you can do is— well, there's a question here. Does the user have— we presume a situation where a user has access to sudo -s, right?

Well, no, I mean, like, how do you either give or deny permission to it? Like sudo su, you could give them permission to the su binary or deny them permission to the su binary. For sudo tag -s, would you just like allow them to do it by putting in the shell thing or?

Well, you can actually in the sudoers command or the sudoers configuration file, you can actually match on command line options. So you can deny that particular a particular one as like a blacklist and everything else is whitelisted.

I'll give you an example of that. Let's see. I got to get a config.

But it's like whitelisted by default, right? I've never explicitly allowed it.

You'd have to explicitly deny it.

Okay.

Yeah.

But the thing here is what— I mean, as an administrator, you do not want to give the ability to your users to just become anything, right? So in the case where you would restrict the commands, you would restrict, hey, you're only allowed to use these commands and then you can't match anything with this particular command line option for the sudoer binary at the top. Block all these. Like there's some admins will create a list of hundreds of options and arguments that will not be allowed to use with the binary, for example.

Okay.

It's a pain in the ass, but I think it's a way to solve your problem.

I haven't done it myself. Okay, so can't speak for him.

Another thing I want to point out is running things as a sudo, is it really the same as sudoing in Teru? A few things don't work. For example, you can't sudo echo stuff into the proc file system.

That is correct. You'd have to use tee.

And the reason for that is because echo is a shell built-in.

Or you can also do sudo bash -c and then put the command in there.

So you're saying it would work to do like /bin/echo?

Yes, I believe so. We'll test it real quick just to be sure. So sudo /bin/echo, right, and then we want to pipe some data into, say, /etc/passwd variant.

And this one, this particular—

I'm sorry, I'm going to do it in my box so I do have permissions to do this. One second. I switched, I forgot.

So, Andy, I have permissions, you're right.

I thought I had permissions, you're right. sudo /bin/bash. Oh, maybe not.

So that doesn't work either.

Why wouldn't that one work? Because I'm pretty sure it was the fact that it was a shell built in.

But like, SSH is a shell built in, and you can still use sudo with it. I'm guessing it's because you're redirecting to—

No, no, it's right.

You're directing the text into the program that's writing to the file.

Great point. Great take on that. You're directing the text— Ah, yeah. Yeah, that's right.

It's an issue with stdout and stderr.

OK, that makes sense. That is right.

OK, cool.

Jon Schipp, I think, wants to run something with elevated privileges, but to make the output be your file, you can do that.

That's right. Yep, that is right.

Oh, so wait, are you saying that right now Bash is trying to take the standard out of the sudo command and do that? So what if you put that part in a subshell or something?

Which part, this part?

Yeah, like the whole command except for sudo.

Yeah, first of all, let's make sure being echo, or just because I have so much. Strivest, sister. Okay, so that does not work.

Or however you do—

yeah, this is kind of like— this was like kind of an introduction.

What you can do, the way, the way that you would actually do something like that though is, um, you'd spawn—

like Devin said, you could do the bash and you can ask it to just do this, and that would actually echo that.

I'm just wondering the reason. I'm wondering if the reason is because bash is trying to apply the stdout redirection to the command and not part of the sudo operator.

Yeah, like this handler by itself. So the sudo runs the command and then gives your text back to Zeesh or Bash, and then Bash writes it to a file. You're the one who's running Bash.

Right, so yeah.

Okay. And when you run Bash, it's elevated permissions. Yeah. Okay. Okay.

Any other questions?

I'm not an expert on this.

One other thing is When you use, for users, they can do a few different options. He talked about the -shell option and that's different from the -i option which actually gives you an interactive shell. And you'll find out that the variable set between these two things are different.

We can actually test that real quick. So, let me just log in.

So, diff sudo -i and we're going to do set. Oh no, you want to do process substitution here.

I believe. And then you want to do sudo -s and then you want to say— can we just do a command like that? Let's try that. I forget. Yes, okay, you can do that.

So you can see here's the difference between the two. The first one actually is an interactive shell and the second one is not. So we have a difference in variables. So let's actually make the output a little bit prettier.

So -u and -w. Oops, excuse me, I scrolled up. We actually have—

okay, so in this particular case where the first one, it looks like the -set, that's what inherits my environment variable for home. Rather, the interactive shell will actually load the profile for /etc/profile and that of the root user, so root.profile or bash_profile in that particular case. However, in the case where you use the -s option, it does not do that.

You keep the home directory or the environment variables of the current user rather than that of the user you're becoming as the shell. Not meant to be—

I'm not sure if it keeps your shell or it starts with a shell of the same type. But if you're in zsh, which is the Zenergi user shell, and you have bash set to be the default login shell of root, s will go and keep your zsh and i will open bash.

Really? Okay, I didn't know that. I never use zsh. Uh, another thing is, uh, so you can use sudo as a particular user.

So you can do sudo -u I want to become Wayland.

Check. Don't we all? Kitty. Kitty. So sudo -u wallander.

That is wallander.

That's a particular command that I want to execute as Wayland.

I can do bash as Wayland. You can see that I am now the Wayland user.

And then I have the interrupt to shell that actually pull up information from the various scripts. So there's that. I did, I said the same thing.

In this particular case, home is still Jschipp and that's what J8S is trying because that one did not invoke an interactive shell that actually called upon the profile scripts that batched it off. So therefore, I am Wayland now, however, my home directory is still Jschipp. There's a few other options that are useful. You can run a command as a particular group. So I can do -g and say wheel, for example, or something.

Also, it's useful—

oh yeah, if you use sudoedit, that's a way you can use an editor like Vim or something. As a sudo user without having them to be able to escape out and get a shell with like, you know, colon shell or the exclamation point and some command if they're that as root. Sudoedit is a program that allows you to do that.

Hmm.

Yeah, the way it works is it lets you edit the file first, not add the permissions of the user, but then it would actually copy that to a buffer, and then once you exit that, sudo then copies that to the appropriate file.

That way you're using the Vim or Emacs RC.

RC, yeah.

Your user instead of the root user.

Right.

I've been relying on that stupid T trick.

Oh, nice. Yeah, I still have that in my Vim RC, but I haven't even used it yet.

I just forget to use it. But see you later, man. Take off.

Yeah, see you guys Thursday, right?

Yeah. Yep.

Do we already turn in the files? I mean, uh, slides for NSM?

I updated both of the slides. I updated for Log, I updated for OpenNSM, but if anyone wants to add anything more to Log, you can.

You did NSM and Log?

Yeah.

So Devin, you're gonna come too?

—right now. Yeah.

No, I meant—

yeah, in the case for like cd, the only reason cd doesn't work is you just execute as a command like that, and then you're automatically back to where you were after. So you don't actually end up in the directory that you executed to.

So I don't know. I guess we're done.

But there was a book that's really great, and it covered actually a lot of that stuff you talked about with with the redirection. Um, it's called Pseudomastery. I just kind of forgot a lot, uh, I forgot a lot of it, but that's why I got that quiz of all those things. I need to go practice that. Uh, but yeah, it's a really good book. You get like $10. Cool. And it's like 100-some pages, so I literally read it in like 2 hours.

So what do you guys want to do, like me and you do Cigna and them two guys do WUG, or what?

Who's them two? Me and Devin and Wayland. I thought Wayland wasn't going. He's going.

He thought it was on Wednesday.

Okay, I was lost.

Wednesday is lunch at 12:30, right?

Man, why'd you pick Perkins? Because it's close and it's cold out. It's disgusting.

I had a pretty good steak there last night. I mean, I know, I mean, for like a, like a $10, $15 range, chicken finger sandwich is pretty good. I don't know, I was asking that too.

Just leave it there and then they'll find it, whoever it was. Okay. All right, that concludes vlog.

It was funny because I said that to everybody else and I was like, oh, but watch, Waylon is going to complain about it.

Fucking virgins. Come on, I can't be the only person. It's convenient.