LUG Meeting (PXE, Vagrant) 02-02-2015

Well, this is LOG everybody, the Linux User Group. Say it. LOG.

Big and heavy as wood.

Go ahead and get started here. So a few updates. We do have a GitLab server now, so if you would like to have your own private repositories or even public ones, we can host it for you. And I can give you a look at what it looks like real quick. We don't have DNS names unfortunately, or at least pretty ones. Here's GitLab. I'm already logged in. So if you want an account, let me know. You'd have created your own SSH key pair. You upload it and then at this particular point, you don't really have anything. But you would have— you can do all kinds of stuff, integration with APIs. You can create your own groups, create your own projects. And so you can create a new project and just, you know, this would be like GitHub or my Git repository test. You can say whether you want to make it private, internal, or public. And then you can assign it to a group. So we're going to be sharing this to OpenNSM. So any stuff that we want to work on with OpenNSM that wants to be private and we don't want to use GitHub, if you have a limited amount of private, uh, password availability, you can put it on here. So say I want to be account or make it under the log group and hit create project, and then I would have it. And then I have the SSH key pair set up. I can go ahead and clone it and push to it, and then we can use that to integrate with all the other stuff like the email and whatnot. So that is that. Also, we have an IBM RS/6000 now that I picked up at a hamvention last year, and it looks like this. It's huge, it weighs so much. Waylon and I endured carrying that up his— it's heavy, bitch— 2 or 3 flights of stairs. It was ridiculous. But, uh, it's like this big, this wide. It's huge. But, um, so that way we can play with proprietary Unix operating system if anyone's interested. So Unix, Unix, yep, real Unix proprietary. Yeah, from IBM.

That's what I learned. I cut my teeth on Unix. When I first came to school here, I had a UX4 account.

Really?

Yeah, there was no such thing as a web interface for email.

I forget how old you are.

It was before Pine even.

Nice. So we can play with that. Linux in the news. So there was a huge vulnerability, one that came out that was called Ghost. You may have heard about it, but it was a vulnerability in the glibc library, GNU C library. And it was a problem in the GetHostByName function which used the name resolution and that's actually a deprecated function. You're supposed to use GetAddressInfo now, but GetHostByName was vulnerable to a potential, or an attacker could craft a string and pass it to the program as an argument and corrupt any particular point of memory that was available in the program and that it was in the specific location the program was in git host binding actually calls a helper function called the _nss_host_name or something like that. And that is the vulnerable program that you can actually overflow there. So more details at the CVE and at that link above.

Yes? The GitLab thing, sorry to jump back again. That's basically like GitHub, but we're hosting it. We have full access to it. We're the only ones that can get on there. So we can have private repos on there without paying and stuff like that?

That's the idea.

Like all the premier benefits of— get without having to pay for it.

Yep.

And then you're going to look at all of our secret stuff though.

No, I'm not. We can have some mutual trust, right? But see, this is really nice. It actually created the repository. It tells you you can access it by HTTPS or SSH, gives you the URL right here, and tells you how to set up Git. You can go— it tells you these are the commands you can type to begin using the repository immediately. So it has directions. So go ahead and configure user key. User account in Git and then run these commands and then you can automatically start working with it immediately. So if you've never used Git, all this and you've got to work with it. Let's get a t-shirt made. Just the initialization, but yeah. This is a good way to get started with adding stuff to a fresh repository. So, how are we going to take a look at it? I'm going to go ahead and remove the project because it's a test and— ad hoc. All right. Posing that. What else do we have? LXD. So Ubuntu, after a little while, has a— well, I shouldn't say Ubuntu. It's a— Linux Containers Group has a hypervisor for— or a so-called hypervisor for containers called LXD that consists of a command line client called LXC and the daemon actually called LXD. The D stands for daemon. And this allows you to work with containers at a level that's better or more abstract than just working with individual ones. So you can actually have a daemon to manage it all. It's kind of like Docker, kind of thing. It's more enterprisey, designed for businesses and people that want to play with them. But they have their own now, and I've been really wanting to play with it. So this is available, but this is not using Docker containers, those are different. This is actually just straight up LXC, the latest from the Linux container people.

So you're saying that this is more enterprisey?

No, no, no, I'm just saying— I would— I just— I probably said that, but what I meant is that it's designed for people putting systems in production and stuff to run containers, not enterprise. Not enterprisey per se.

But is this open source?

Yeah, yeah. This is from the group of people that actually created Linux containers, the LXC group.

Oh, really?

Yeah, so, you know, you've used LXC-star and all those. It's these guys. So, it's just a better way to manage multiple.

But is it— what's the learning curve like on this? I don't know.

I haven't used it yet, but I hope we get to play with it. So, I'm going to try to work with Jon in my free time. Next. So, I actually forgot to look at this, but Docker 1.6 came out with the ability to work with Docker. Vagrant came out, 1.6 came out with the ability to work with the Docker Vagrant environment. So, I would watch one of these videos. They're so long, 4 minutes. But this is a way to quickly spin up Docker inside Vagrant. So you can automatically have it provision, configure the VM for running, for having the Docker daemon running so you can connect to it from the client, etc., etc. Check this out. We'll be talking about Vagrant more later tonight because I'm doing a talk on it. But for now, you can actually watch this after the talk. Maybe we have some time. Next, we have LXC also getting containers. We do a lot of containers in here. At least I know Shane and I and Whaley now are pretty interested in them. So they're pretty cool. So we'll be talking about a lot of these in the future meeting. But LXC Tools 1.1.0 was also released. And it highlights where here is the ability to do checkpoints and restores for containers. Serialize the container state to disk. Do you see that, Shane? Wait, what? This allows us to serialize the container running state to disk for live migration, later local restoration. This might solve your problem, I don't know. Just something to think about. I don't know, just from reading it the way it sounds, it might be what you're looking for. We had the one comment.

I mean, like, just the way that this word— sentence is worded, I'm not sure what they mean by serialized.

Yeah, I'm not either, but it's potentially—

but yeah, like, state to— yeah, I would like to—

so we should look into this. And also the new— this is 1.1 that just came out of LXC tools. And then also it has support for running systemd as the init system inside the container now. And init scripts have been updated so you can use the LXC bridge, etc., etc.

Awesome.

There's a bunch more details on that if you're interested. Be able to totally play with that. Also, I found this website for Education Epoch. This is where we talk about good resources for learning Linux and related. And this is a MOOC list website. Crappy— well, not too bad— web design interface or whatever, but they actually have a list of all the MOOCs related to Linux, and of course the one for the Linux Foundation is in here right here at edX. So, You can check these out. There's a few other ones here from— one on operating systems that includes a discussion on Linux and this Linux fundamentals one from WMA, the World Mentoring Academy. So check these out. We're going to go ahead into proc period. This is where we talk about the proc file system or the kernel file system that is a way you can interact with the kernel through a particular— or through a file system called procfs. And this particular one here is talking about automatic stack layout randomization. This is a security feature that was introduced a while back that allows the stack space to be randomized so that if there's an exploit, it can be in a different location every single time from the program is run at runtime. So you can't run an exploit and expect to have some particular instruction executed at a particular place because it's randomized. So it offers a protection in that regard. And you can actually— it's enabled by default on your modern Linux internals, but here's a way you can turn it off. So let's go ahead and fire up a VM. Let's do— and actually, I got Vagrant turned on. You'll be seeing I use Vagrant for a lot of my stuff. I'm going to go ahead and increase the screen size here. Just a moment. All right, now create is— go ahead and bring up that virtual machine. So bring up a brand new virtual machine using Vagrant, which we'll talk about later in the talk. Make it a little bigger as that comes up. So actually it's pulling in a base box, which is a container image and some information describing that, like metadata, and then it's going to go ahead and set up networking for me. It's going to copy any files I tell it to, run any programs I tell it to, all of that is written in the Vagrantfile and makes provisioning a new system very, very simple. We'll be able to log in in just a moment after the machine comes up. And I'll try a few times, well, try to connect via SSH but the system isn't ready to connect yet. It's not waiting, it's not accepting connections but it will in just a moment. Alright, machine booted and ready. And we're now able to get into the system by using Vagrant SSH. I'm in a brand new Ubuntu system and what I'm gonna do is become root and we can check this out. —it asks me for— So what we're gonna do first is we wanna find— take a look at this file. So we can set it to the value of 2, which means it's on for everything that it could possibly be running with it. And what we wanna do is we wanna change the stack to 0, or the setting to 0. And now if we cat that file, you can see the value is set to 0, so now there are no memory protection in regard to the randomization right now. And this is bad. So another thing you can do is you can use this tool that I've never used before that I saw when I was researching this. Setarch. And it actually lets you set specific flags to— on the architecture per program. So you don't have to do it system-wide. And you can see the -r here is the address no randomization option. And it actually disables the randomization of the virtual address space. So in this particular case, if I want to run, say, ls, then ls with that off, that would have ran it with that randomization off. As well, if you want to do it persistent within the kernel, then what you would do is that you would use this in the system control script because almost every modern distribution has this file system control, the system control program reads on boot to actually apply specific things to your kernel. It's another way to interact with it so you can actually just place This here, this here, and then what we can do is we can actually apply it to the kernel and just say -a, oh no, it's -p, actually forget it, let's try it. I think it's just -p. Yeah, it's -p. Okay. It's -p. So -p and then the file that you want to read from and then now you can see that it read that file and it set it to 0 anyway. Of course we already had that set to 0. So that's one way you can do that. I was actually going to write a little C program to actually print out the memory address but using the address of operator and show you before and after but I actually didn't have time to do that. So did not, but that's a quick way to demonstrate it. Before and after you set the option, you'll see a different— the first one without audio, you'll see the same memory address for that particular variable that you want to use, and then without it, or with it, without it you'll see the same address, but without it, every time you run the program, it'll be different. So FYI. And then we got SSH section up now. Did you want to talk about it from here or— Yeah, that's fine.

So basically I just added a couple of simple yet useful options.

You can use in your SSH daemon config file.

The first is the address-family directive and it just specifies what version of TCP/IP that your SSH daemon is going to use. So, for example, if you're in an environment that has a whole lot of IPv6 or no IPv6, you can specify whether or not to use IPv4 or IPv6, or if you don't really care and you want to remain agnostic to it, you could just use any, which is the default. Default on a vanilla SSH daemon config. Also, the listen address directive allows you to differentiate which IP address is actually providing SSH services to the internet. So, for example, you may have a host that has several different IP addresses, but you don't want a daemon listening on each one of those. You can specify which of the IP addresses would have that.

Alternatively, what you could do is you could—

if you have a host with a single IP address, you could just specify that IP address but a non-standard port in this directive to have the one IP address listed on a different port than 22.

So yeah, pretty simple. It's in, under networks when you're in system admin. If you're not using IPv6 and stuff, like automatically, if we're in environments where we're not using that to communicate with different services, I'll automatically go ahead and set address family to inet so it uses the IPv4 family so that we don't have to worry about any issues with the firewall of any access controls. You'll find a lot of people over in iptables, but they forget that iptables doesn't work with IPv6. 6, it's IP table 6 or— yeah, IP table 6, you need to do that. A lot of people don't configure that, so just setting your daemons to actually listen on the— on the— the protocol that you actually— the network protocol that you actually expect is recommended, so.

Oh, and so the other thing is that I didn't mention but it's in the notes, you can use as many of these keywords as you would like. So for example, you could put 10 of these with a single IP address and you could have 10 different ports listening for SSH, or you can— like if you have multiple IP addresses, you could use any number of those keywords up to the number of IP addresses that you have to specify which would be serving SSH.

Cool.

So these are options that you're adding on when you're trying to connect to a remote server?

No, that's a good question. I probably didn't clarify very well, but this is in your sshd config, so if you go to /etc/ssh/sshd_config, sshd_config, you would just add these options to that file and then restart your daemon.

That makes a lot more sense then.

So, and depending on your flavor of OS, you could use, like for a Debian or Ubuntu install, you would just say service ssh reload, or for Red Hat, it would be /etc/init.d/ssh restart. So, anyway, nothing too terribly complicated.

Cool. Yeah, so you're saying you can listen on multiple ports for SSH requests?

Is that what you're saying? Yeah, you could do that with these keywords or with that keyword. There's another way to do it too.

And could you shut down like standard port? You know, like say you still want to allow SSH access but you only allow to people that know to look for it on, you know, 1337 or something instead of 22?

Right.

So you could not run it on 22, and you could just run it on any number of random ports. Obviously, you could even run it on other well-known ports, like 25 or 21.

So would you put, like, an address, or then you write the port and then put false or something? Or how would that—

No, you could just say 192.268.1.1.

:20 or 21.

You're telling it to listen, but how do you tell it to not listen on a port?

You just don't listen on that port.

So it'll automatically have 22 in there or whatever?

Right. That's the default, yeah. Right. And you can also patch it on the command line too, sshd -p, I believe. That's cool. All right, moving on to shell space.

Oh, but I guess the other thing I was saying is you could run it on the default port for SMTP or anything.

But obviously that could cause conflicts if you tried to bind to that port with another service. But you could do it if you wanted to.

I was just thinking of like— it's not a great mitigation, but just against obvious SSH attacks.

It's an obscurity tactic. If someone was actually sending TCP traffic to those ports, they would be able to know what was listening there.

And then you could also do port knocking on top of that. Great. So, it's a little, yeah, that's interesting.

Alright, moving along to shell scripting. So, in this particular one, talk about some helper functions I use whenever I'm programming in Bash. So I use this one called "hi" and it's just a way to, after a the command returns true to say something that lets you know that it met expectation. And then the opposite of that is what we're gonna call die, borrowed from Perl's die, that if the exit status of some program executes as greater than 0, then we'll usually execute this function to do something like send an email or write to an IRC channel or print a message with cowsay or by itself. And in this particular case, we'll show you how it's used. So let's just look for example of die. Okay, so we check for this kernel variable is set. Then what we do is we do wget, whatever is set to this URL. So it goes grab the kernel from this URL. But if wget fails and it returns something greater than 0, then this gets executed with the or. So die then takes control and it says download of kernel failed. So very simple way to print a nice little message. And if you go back up to die, what we can actually see here is when it first does a check if cowsay exists, and I have cowsay at the top, you can see very top highlighted, so it looks in that path and if it's not there, then what it's gonna try to do is it's gonna echo to your screen And what we're using here, we're using a shell variable here, the dollar sign and the asterisk to match any arguments that are passed to the function. So in this case, it would be the string download failed. So it matches anywhere all the way across for that string and then it prints it out. Then what actually happens, whether CalcSays is installed or not, what actually happens is it will write to IRC if it detects that the IRC client is at the top, is IRC say. And then finally, it'll check if your mail program is installed. And if it is, it'll try to echo or send that text to your system. Or the email address you provide above, I mean. So I have— you have to fill out user@company.com by default, so you just change it to whatever yours is. So there are two little helper functions that I use in my scripts. And then finally, what we have was this one I wrote to do provisioning of packages. I call it package check. And a lot of times when you're running scripts over an already provisioned system, some packages may be installed, some may not be, but it'll go through and try to enumerate the entire list. It'll take a little bit before it actually installs them if they're there or not. If they're not there, it won't install them. So I wrote this little script, this little shell function that checks for the number of packages given in this packages variable, and then it runs it based on the operating system that you are running. Debian or Enterprise Linux would be CentOS, Fedora, etc. It tries to count, get the count of that and compare to the number that you want to install. And then if the count is different, it actually installs the packages that it needs. So that's just a quick way to not have to go through the entire list every single time. And basically there's another function called number_of_packages that you run and you basically— an argument is the actual number of packages and they'll try to figure that out. So I run it right here. You can see that packages here, this should be a number, you know, whatever that is returned, and that sets a package count, and that is used to actually do the conditional here, whether the count of packages on your system is different than what needs to be installed, and it'll, it'll execute appropriately. And the other cool thing about this little function is that it'll work on a Debian based systems or the enterprise. So you can give it any list of packages right here. So I can say I want to install cowsafe, buildessentialflex, python, and a few other libraries, and no matter which system I'm on, it will actually install the appropriate packages for that. So it's a nice little helper function. They're fun to play with. We can actually do— let's do I'll show you. Let's do hi. Go ahead and do that. It'll pb— pb paste into hi.txt. Source hi. Then I'm going to do hi, this is a test. You can see that it just echoed that value. And it doesn't have cows— it doesn't know where the location of cowsA is. But if we set that, So cowsay, where is cowsay? cowsay is /usr/local/bin/cowsay. So we can go to hi.txt, we can go up here and cowsay is equal to /usr/local/bin/cowsay. At this point, we'll source the script again and what we'll do is, hi, this is a test, and then it runs cowsay, this is a test. So I like to have these, I think it's kind of humorous and not so boring if you incorporate things like this. In your scripts. But in the case where cowsay is installed, it will have like wget failed in the cowsay box. I don't know, I kind of like it. And then die does the opposite. Die actually uses cowsay with the option where his eyes are crossed where the cow is dead. So that's just got a little small difference. So it's just fun to play around with. So there's that. And then finally, oh, okay, sorry, not finally. vim-vicinity is next. This is where we talk about little things Vim, share tips and tricks. And let's see where I want to do my next thing. I guess we'll just do it here. So there's a few faster moves you can make. I used to, before I knew this and I wanted to create a new line, I would go to the end of the line, press A to go to the end, and then press Enter. And then that was 3 different keystrokes. But you don't really need to do that. You can just press the letter O and it will go ahead and create the new line for you. Another one that is useful to do the same thing before the line or the line above is capital O. So it goes ahead and you can just type capital O and it just saves you a few keystrokes if you can do that. And the other one was the F, which is a search, uh-uh character. So you can do this and it will jump right to the character that you want. So type in F the letter capital P, it jumps right to the location I want. F to capital S jumps to the location I want. S capital N jumps to the location I want. And that's just a quick way if you actually end up being on a line and you don't want to have to go all the way across like that or go here then go back, you can just use that to find the character that you want. I just want to jump to K. You can see how that's much faster than going like this to the K or much faster than going like this to the k. So fk, I'm at the k, I can begin editing the stuff right there. The other one was the directory listings in Vim. This is actually really cool. You can use Vim to actually browse the directory structure for files. So I'm gonna use Vim, I'll go ahead and open it up, do colon e and then dot, and you can see it's actually the directory structure. So I can then pick which file I wanna load. And I'm going to load newiso.sh and it actually loads it for me, so that's really nice. So I'll often, when I don't know which file, instead of doing ls, when I know I'm going to use vim later, I can just go ahead and do vim into that directory and I'll be fine. You can do the same thing and specify the directory. So let's do /usr/etc or vim /etc and I can go through my /etc directory and pick which file I want to edit. If I want to edit the profile script, I can edit that. Another thing was using Vim. Vim has the ability with the plugin that it comes with to actually open up tar.gz files. So you don't actually have to do tar.gz and then— or tar— excuse me, to actually list the contents with VTF. You can actually just look at them in Vim and actually edit them directly in Vim without actually untarring it, which is really badass. What we can do is I'm going to find— I don't really know where they are on my system, but let's just look for anything just .gz, but oops, in my home directory. It looks like there's one. There's a bunch of other ones. Let's go ahead and take a look at what looks good here. So much crap. Let's just go to this one, right? Well, I'm sorry. Let's go to this one right here. Oops. So I specified a file and this is what's in this particular direct— in this file, tar.gz file, and I'm actually able to modify it from in here. And where it's zipped up and there's more than one file, let's do this again and get rid of the Bro stuff. I probably should— one second— I should search for TGZ actually. That's a better match because it's an archive rather than just a GZIP file. So here we go, bench.tgz. So let's go ahead and look at that one. So this is— there's multiple files in this archive and it's GZIP'd.

Didn't find something.

And here they are, all these files. This is the directory that the files are in, in the yellow. And if I want to edit killgult.sh, I can just press on it and it actually loads that file in there. So it's a really quick way to browse what's actually in a tar gz file.

That would just be like because somebody updated it.

Okay, and then, uh, I guess that was really it. And then we can take it, uh, go ahead and do, uh, C Corner with Devin. Hey, hey, uh, you join here? Yeah. Okay, great, excellent.

Does C work on Apple? No, I'm just gonna— well, it doesn't probably, but that's just— oh, this is FOCICS compliant. I know, I know, I'm kidding, I'm kidding, but Actually, let's use this.

Use that.

Does it have your— does it have your vimrc in it? No. OK, well, I'll just SSH into EWS. OK, cool. Be careful with your wildcards. Yeah, I wouldn't want to break anything. Might break the network. Okay, so today I'm going to be talking about system calls, and first I will explain what they are and why we need them. So on a lot of modern CPUs, they have some sort of security model. For example, on x86, we have the rings model. So there are a multitude of different rings. You can imagine them being concentric circles. Outermost one, or ring 0, allows unrestricted code access. And as you go towards the center, you lose access to certain things. Like ring 0 is usually reserved for the kernel. And then intermediate rings are for things like device drivers. And then finally, the very center ring— I think it's ring 3, but don't quote me on that— is used for application code. So let's say your app— I mean, if we didn't have those, it would be really bad because, for example, a user application without any privileges could just, for example, arbitrarily access the hard drive or the frame buffer network card or anything, which means that things like file permissions would be meaningless because you can just access the bits of the hard drive. But on the other hand, the applications do have a legitimate need to access certain files. So that's what they use system calls for. It's for the OS to provide a well-defined API with security checks to access certain functions in higher rings. So that's, yeah, that's why we need them and how we use them. In reality, system calls are, you would write them in assembly. But that's accessing it through the ABI, which is very error-prone because you have to manually specify the calling conventions and all that stuff. Plus, it could change between different versions, different platforms. So Linux provides wrapper functions, and they're the preferred way to use them. There's also— OK, so just open a test file. And for example— oh, and how to find information about system calls, we will use the man pages, and they're on page 2. So for example, man 2 open will give us information about the system call open. So—

There we go, including the stuff we need to include.

So, yeah, so today I'll be covering a few basic ones: open, read, write, and close for files, and then fork and waitpid for forking and dealing with multiple processes. And then you can, you know, read about the other ones.

So—

Are you talking about 704s at all? No, no, because this is multi-process stuff, not multi-thread. Point. I can do multi-threading in a separate one, though. Okay, so. We will— okay, yeah, and so with the standard C functions like fread, fopen, fwrite, fclose, stuff like that, they use file pointers. So that's something that's defined in the C standard, and it'll work on Linux, and it'll work on Windows. But system calls don't use file pointers. They use file descriptors, which are unique to the Linux kernel. So you code using Linux system calls won't obviously work on Windows. And unlike the complex file pointer structure that, you know, points to some area of memory with a bunch of information, a file descriptor is just an int. So, and there are a few of them with a special meaning, as we'll see. And, okay. So first, I will check if it includes more headers, because it would be very much a— OK, so the first one I'll do will be open, which is used to open a file. In C, you would normally do this with fopen. But this is a lower-level call and, you know, fopen on what is compiled on Linux systems ultimately does make a call to open. It just abstracts it away from you. So if you look at the man page, you first specify the path name and then flags. That's the basic idea. And we go down here. We have things like O_APPEND, asynchronous, create, Hey. Yeah, so the layout is different. Okay, so here we will be creating a file. And actually, first, before we do that, let's look at the close system call. It's really simple. All you do is specify the file descriptor and that will close it. On error, -1 is returned. And the file descriptors that are already open that you should be aware of are 0 for standard and 1 for 1 for standard out and 2 for standard error. So let's see, first of all, let us close standard out. And then we will open a file, let's call it log.txt. And, uh, So create. I don't know, maybe we need to specify additional flags. And then we can use now— and I guess I should be checking the return values, but I'm not going to bother since that's too much typing. So we'll close stdout, we'll open a log file, and then we will use, instead of write, we will use our standard, uh, printf or puts or whatever.

So, hey, why do we need to close stdout?

Well, we don't, but this is to demonstrate something about file descriptors, which you'll see. Okay.

Aimee.

Okay, so Let's see what happens when we run it. Apparently, bin created with read permissions. Have to look at the man page. Open. Yeah, I'd probably have to add some more flags in there, but I'm too lazy, so—

See if we can— there you go.

Grab. OK, well, what I was trying to demonstrate is that when you close stdout, the file descriptor will become available. And when you open a new file, the system gives you the lowest currently open— or I mean, not open— the currently available file descriptor. So if this had worked— and it would take too long to figure out why not— a regular printf call without specifying find which file or anything, would have actually written to the log.txt file instead of standard out. Oh, well, actually, obviously that one. So, yeah. Yeah, so you know the file descriptor of log.txt is actually 1 because is that's the value returned by open. And all puts or printf does is that it actually writes to standard— it writes to the file descriptor 1, which is normally standard out. But in this case, it would actually be writing to log.txt. So there's probably an issue with the permissions or something. Yeah, it's not letting me read it. Anyways. Yeah, with the proper flags to open, that could probably be avoided. So yeah, that's— so next I'll just do read and write. And for this we'll implement a cat-like command, which will— but instead of reading from a file and all that stuff, we'll just read what's read into stdin and then output it back to stdout. So first let's create a buffer. Let's, for example, Make it 4K for the heck of it. And then while read, let's look at the man page. So we specify the file descriptor, a pointer to the buffer, and the amount we want to read in. And then on error, it returns -1. So while read— we'll be reading from stdin, so that's 0. We'll be writing it to buf, and then we don't want to read past the size. While this is greater than 0, we will write 1— we will write to stdout, which is 1. And actually We need this variable because if the buffer isn't completely filled because we've reached the end of file, we don't want it— or a newline or whatever— we don't want to output the garbage past the end of this. So we will write 1, uh, So as you see now, it functions like a normal cat program, and by default, the terminal is newline buffered, so it won't do anything until we actually hit this, and then when we hit Ctrl+D will detect an end of file and exit gracefully. So that's read and write. You can do the same thing with files. Just replace 1 with the file descriptor returned by open. And yeah.

What was the bytes read for again? OK, so bytes read.

Bytes read, yeah. Yeah, my bad. Well, actually— Pronouncing it that way helps you understand a little better what it actually is doing too. Yeah. It's a buffer that holds the bytes that have been read. Yeah, so read will return the number of bytes that it actually read. And let's say that we don't read 4,096 bytes. For example, what if it's newline buffered? It'll read how much there is until the newline or an end of file that you send to it. Instead of bytes read, let's actually see what happens when we try to output the whole—

OK, so this is actually— We get zeros or garbage data?

Yeah, we'll get garbage because it'll try to output the— unlike string functions, write, you have to specify the size of the amount that you want to write it out.

So it'll be whatever's in memory, right?

Yeah. And since this is on the stack and it has automatic storage, there's no guarantee to what— it could say, for example, go Seahawks, for all I know. Just here so you don't get fined. So I hit test. Notice it does output test, but then it outputs the rest of the contents of the buffer up to 4K bytes as well. So whoopsie. I obviously don't want it to do that. Equals 99. And, uh, yeah, let's see. And it actually screwed up the terminal, so— Clear.

Type reset. Yeah, that's what I did, but— Oh. Yeah. So, tput space normal? Control-C, do things.

I did do Control-C. I don't even know whether I'm in or outside of this box. Use the wildcard. I'm just going to close this tab and then open a new connection, I think. Probably the safest way to go. Nope. He just dropped his whole file system. Oh yeah, it did. It's okay. Oh no, you were on EWS, right?

Yeah. So can you do, uh, Apple Shift Plus? Increase the terminal size. Oh, but okay.

Did you see this DShell, uh, forensics framework? Yeah, we really— it's very bad not to keep track of the number of bytes that you've read in. OK, now we will do what I think one of the coolest things. I'll do fork, waitpid, and exec. So those are incredibly useful. For example, just today for one of my classes, I was writing a stripped-down version of make. So it goes and it reads reads the target and executes all the commands under it. But the exec command will actually replace your process image in memory with the image of the process you're trying to execute. So that's— if you just have a single process, it's going to completely wipe out what you had before, just run that command and then quit. So you actually want to have it spawn a new process, a child process, and then have that child process run the and then report its status back to the parent process. So you can do this. And, well, first I'll just do a simple demonstration of fork, which will spawn a child process. Are you going to fork-bomb AWS?

Maybe. Maybe.

Well, we're all witnesses that it was an accident.

Yep. So— It's a good thing to know.

You should actually teach us this so that we know What not to do. You're supposed to put while 1 fork, right? Well, yeah, but the idiomatic Seaway is more semicolon semicolon. But yeah, then you would. Yeah. So don't do this. It could exhaust some resources. Well, especially if you throw something in there to actually do. Yeah, like computing pi or something. Right? We'd take down the network in what, about 4 seconds? OK. The syntax for fork— actually, I'll show you guys something really cool. So first, I will have it print a statement, "I am process," kind of like "I am Groot." And then we will— and mommy. Yeah. So if fork equals 0— so how fork works is that in the parent process, it returns the PID of the child process. And then the child process, it returns 0. So that's how you can tell which process you're in. So if— so if now if we're in the child process, we will now want to print out— so we will print out our own PID and then the PID of the parent process. Of our parent. And actually, I should— yeah, so, and note if our parent process had done getPPID, it would probably print out the PID of either of the shell, or we could actually kill the shell and have it be acquired by the initial process. But anyways, Now just by looking at this code, you would think that I am process would be printed out once, right? Just by looking at this code because it happens before fork. So let's see what happens.

Oh, the fork is the same copy, doesn't it?

Yeah.

Yeah. OK.

Oh, but apparently it printed out twice. Can anyone tell me why that is? Yes? You were in that class, yeah, so you don't count, actually. But yes. So— 241? Yeah. Because of race conditions? Well, no, it's because printf— by— we'll put the thing into the terminal buffer, and by default, stdout is newline buffered. So, and then when you fork, it creates essentially a perfect copy of the process, so it'll actually copy the buffer. Then when you call printf the second time, it will— first it'll flush the— it'll like append this to the buffer and then flush it once it reaches the newline. So since I am process was already in there, it'll be printed out again. Now if we added like a -n or if we added a newline here or we put in an additional command to flush the buffer or anything like that, then— I don't even know if newline will necessarily do it. Because stdout is newline buffered, so a newline is guaranteed to flush the buffer. You can change that setting too programmatically. You can For example, if we printed this to standard error, we wouldn't have this problem because standard error isn't buffered. It prints out the characters. So here now, it will only print this out once. No, print it twice. And then we can do all sorts of cool stuff with this. And for example, yes, Professor Angray mentioned this, but I'd seen it before. There's an algorithm on 4chan called called sleep sort, where it takes a list of numbers and it forks a bunch of processes. Each one is assigned to a number, and each process sleeps for the number of seconds equal to the number and then prints it out. So you'll have it print out all of the numbers in the list in order in O time. So it's a very neat algorithm.

Okay, and—

It's actually pretty good. Yeah. Only problem is when you have like 1, 2, 3, a million, it might take a while to complete. But yeah. So yeah, so that just tells you that linear is not always the best. Yeah. So for example, if we have something like this, then we can fork it. Now if we're in the child process, we're going to run exec, and there are actually several variants of this. Exec l is one of the most frequently used. And for example, so then the first argument will be the binary to execute. So we could put in something like /bin/sh or let's just put /bin/ls, for example. And then the next argument is what is received as the first argument by the process, which is by convention its name. So we could pass in anything, but usually we want to pass in the same thing, which is /bin/ls again, or some processes might get confused if you do the wrong thing. And then, for example, for arguments, let's do -lh, for example, just for the heck of it. And notice, so this call will— Unlike, for example, calling system, this will actually replace the image with ls. So once the child process terminates, this— the program will not continue running in the child. It will also terminate. So the benefit of this is if something goes wrong with the call, we don't even need a conditional. We can just assume that if it gets past the point of exec, it's failed, and then return the appropriate error. Otherwise, we are going to wait. So there's actually— wait is a special version of waitpid. So man to waitpid here. Wait for a process to change state. So we know the PID of the child process, and if we just have wait by itself, we have a pointer to an integer variable called status where we can read them, and then From there, we use some macros that are provided for us, like did it exit? What was its exit status? Was it terminated by a signal or interrupted? Did it core dump? All that stuff. Was it stopped? So that's— we can use those macros to access those fields of status. And so we actually Let's replace this with this PIDT, PID equals George, and then if PID equals 0, then we will execute this, or else we'll have a status variable. And then by default, if you don't— let's go back to the man page. With waitpid, you have to pass in the PID of the process you're waiting for in options. But in wait, by default, it waits for all child processes. So in our simple example, there's only one child process. So we can just do this. And then assuming all that is good, we will exit successfully. Forgot to include a header. Oh, and, uh, of course I forgot to include the actual directory. It's just And there we go. So it prints out— it gives us an ls of the current directory. And we can use those macros to check things for like the return value. So we can see whether the return value was 0, non-zero. We can check whether it was terminated by a signal.— or did exit itself gracefully, all that fun stuff. So that is fork, exec, and waitpid all at once. And yeah, so you can also do cool stuff with open, read, write. And you can look in the man pages for plenty of others. There's also, for example, the kill call, which is the equivalent of kill on the command line. It's a very sadistic term, but it just means send a signal to a process in general. You can specify the type of signal, so you can use it to kill other processes or tell them to do stuff.

So yeah. And that's it. Any questions? Where did you learn this stuff?

Well, I mean, I'm in a systems programming class right now, but for the most part, I think I understand it since I did do a lot of C programming in my spare time. So yeah. Cool. Devon's a prodigy.

When you were starting out, what kind of things did you do or spend time with?

Well, here, I'll show you an example, actually. This is RC4 with a modified key schedule actually that is safer than the default by how much, no one knows. But I just basically, I'll show you, like I can show you it's used for encrypting and decrypting things on the fly. Like this is the actual algorithm and then this for example, We will here, like the usage encrypt decrypt streams, it reads something in from standard in, encrypts it, and prints it out to standard out. And you can do cool stuff with this. For example, this is kind of a contrived example. So when you do, for example, curl google.com, I mean, you could also do this with a porn video or something that you don't want to ever touch your hard drive unencrypted. So curl google.com So first, okay, well, I don't have it in my bin on here, so let's see what the usage is. So encode options key for, so for example, curl google.com, let's add to that /encode. Let's make our options -n 20. This is— it modifies the key schedule to remove vulnerabilities in the original version. Let's make our key cheese pizza, and then let's pass this into base64 to encode it, and let's write it to a file called taxes because that's always innocuous and no one looks there. So we run there, and then if we view taxes, it's just going to be a bunch of Base64. And so now if we pass it in, first we decode it, Base64 -d, and then ./encode -d -n 20, our key, which is cheese pizza.

I did not spell this correctly.

We will get back the original thing. So this is actually— I tested the throughput. It's in excess of 1 gigabit per second because it's written in C. So you could actually do something like download, have wget or curl or wget, I guess, stream— let's say stream a video from a website and then output it to T, for example, or have it so that You pass it through VLC so you can watch the video while it's streaming, but at the same time pass it so that it saves, but already encrypted, to your hard drive. And then when you want to watch it, for example, you can just cat the file into this to decrypt it and then stream it into VLC to actually watch it. So the unencrypted video never actually needs to touch your hard drive. So this is one of the projects I worked on before I actually I took a C class. Yeah. And then I did some other stuff like writing tools to access APIs for websites and writing bots, for example, to troll people. So yeah. Did you use that to actually hide porn from your parents at home? Well, no, I didn't because I had no need to. Like, it's not like they could get onto my But you were able to say we're trying, so it's nice. What year are you? Freshman.

Ask him how old he is. How old are you? I'm 17. Bam.

What year are you actually? Senior. Thank you for asking. Yeah, so he's a redshirt freshman. 17-year-old senior. Yeah, so I wasn't really kidding when I said that thing about the president.

Are you a senior in college? Yeah. On numbers? Wait, by credit hours or by how close you are to graduating? By what? By credit hours or how close you are to graduating?

By credit hours. I don't actually have all the requirements. I think before the semester started I had 93, and by the end I'll have like 111.

How many AP classes did you take? 3. Community college credits?

Mostly from ISU, actually.

I did take a few community college classes, but yeah. What's your favorite kind of pizza?

Uh, well, I mean, I don't really like pepperoni, I don't really like, uh, you know, Hawaiian, so I'd have to I had to go with cheese.

Would it be safe to say that that would be an encryption key you used?

Yeah, I think it would be, but not in production, just for testing. Did you figure that out? That no write-up thing? Oh, I'm guessing.

All right, and the Smalltalk I have is on a Pixie boot server that I've been working on, and I'm going to demonstrate it, how to use it live. So does anybody know what Pixie boot is? This is really cool.

So yeah, sorry, do you want me to answer or do you want to answer?

Yeah, anybody answer.

Well, it's like it runs a process before boot, right?

Yeah, you— yeah, so Pixie itself is a bootloader, and then you can— you tell it to actually load something else. So it's a way to do unintended installs of Linux. And so I used again, I created a Vagrant VM and called— Is that an eMachine? Go ahead and— EEE, yeah. PC. One of those. I say this is your only PC here. Sorry. But I'll go ahead and pull up my repo here. And I did two different ones. I first did the regular PXE one and then I did a PXE multiboot server. Let's do the Pixie Kickstart one. This is a Vagrant setup, so you can actually, if you have Vagrant running, which we'll talk about next, you can clone this repo and have your own ready to go. So first, let's talk about what it is. So what this will do is the script will download the latest Ubuntu 14.04 LTS, the ISO image that you would burn into a disk, right? What it will then do, it will configure DHCP to give you an IP address, and then it configures DHCP to actually pass you, once you get an IP address, the pixielinux.0 file right here. And this is the actual bootloader. So once you get an IP address, you actually get this file. Your system will boot it, and then what actually happens is that file will have a list in the menu, the VESA menu, C32, from Syslinux. What actually will happen is it will tell it where to then boot the next thing you want to boot. In this particular case, we want to boot Ubuntu server. It tells you where the kernel is. It's in this directory that we downloaded and put it to. And then here are the options that you actually pass to the kernel. And it's just IP addresses and stuff of my system. So, problem was I'm tired of carrying around Linux on CD-ROMs, burning them, or DVDs and USB sticks. Every time a new one comes out, you gotta burn it again, you gotta copy it to the USB stick. It's a pain in the ass. So, why don't I have a way that if I always have my laptop with me at all times, I can easily boot a system. And that's essentially what this does. So we're going to go ahead and— oh crap, I can't actually disconnect from the network, so I'll lose my video feed. Hold on, let's see if I can do Wi-Fi. You'll still lose it on the switch over. Huh? It switches over. Hopefully it does it smoothly. I've had cases where it didn't. This time it hasn't. But, so at this point, whenever you actually boot up this virtual— this VM, any network that you're on, you're gonna start handing out DHCP leases. So if everybody in the room had Pixie ready to go, I could boot them all in my image. So typically that's not enabled by default though, and the system has to be in its boot process. But in this particular case, I can— anytime I want to have my laptop with me, I'm gonna work on-site at a client's or something or at home and I don't wanna burn anything to disk or on my USB, I can literally just category plug Ethernet cable into my laptop and into the system that I want to install Linux on and then turn it on and just let the PXE boot client and then it will actually be able to install Linux. So let's see what happens with this. We may lose video. I hope not. 3 participants. No. Okay. So, plugging— wait, let's hold on, switch— alright, sweet. So now what I'm going to do is I'm going to take this and plug it into my laptop and then I'm going to go to my PixieBoot system and this is the multi one right here. Vagrant status. I think the machine is off right now. Power off. So it's off. Now let's do vagrant up to turn it on. And let's go ahead and boot the machine. And this machine is going to be my Pixieboot server. So I'm carrying around the virtual machine on my laptop. So I've made this in such a way that you guys can all download it and use it yourself. I did all the hard work already. I even have it so that you can run— I wrote a script called new ISO. So you just do new ISO and you give it some command line arguments including the ISO to download that you want to provision. It'll download it, it'll unpack it, and it'll create a Pixie service ready for it to go. So it's pretty sweet. And then I'm doing one for other styles where they're just tarballs of Pixie. You'll find that to be like Kali Linux, if you wanna have those, there you go. They released theirs in a tarball. I'm running a new distribution called new-pixie that actually will do that. So just a moment, that'll be on, that'll be ready to go, and I'll fire this guy up. And this guy should be able to boot, and I'll show it to you in just a moment. Just give it a second, it's booting. And then we'll talk about a little more about how it works. The condition is that that other laptop has Pixie on it. Has a Pixie client on it, yeah. One thing about this, most of your modern laptops will have that already in the box. BIOS. They're all configured with that usually. I've never encountered one that did not have it. Oh, okay. So even this little dinky $200 triple-E PC has it on here. Wait, so you said Pixie is a bootloader? Yeah, well, then what's like Dummyboot or Grub? Grub is a bootloader as well. I don't know, I never heard of Dummyboot, but Lilo and Grub are both bootloaders as well, yeah. But this one's designed— this Pixie boot program, like the actual file is PXE Linux 0.0, that is actually designed to do network installs. It's a very small bootloader. It doesn't have the features that GRUB has, right? It's very small. It's just enough to say you can tell it where to actually get the operating system, where you can get the actual kernel, where you can get the actual file system. So it doesn't have it all itself. You have to go out and download it, and it's smart enough to be able to do that.

And this won't run on the new Windows bootloader, right? Huh? Will this work on the new Windows bootloader? What's it called? EFI? Yeah, EFI, yeah.

I don't know, actually.

I think it might if you turn off secure boot.

Oh, that maybe. I actually don't know much about that. EFI, I don't know much about it. So you say new machines. How new is new? Oh, I mean within the past 7 years, probably. All your servers will have it. You should try it anyway. I'll just show you how to pass. I only have old machines. Oh, well, I mean, there's a good chance they'll have it too. That's just a guess. I don't really have any— Substantiation. Yeah, but, um, your thing doesn't seem to be working. Yeah, it's gonna come up.

It takes a minute, but this is too tall. I gotta go, my homework team needs me.

If we stay up here, lock up.

Yeah, you better. I'm landing on desktop. Can you make sure it's locked up? I gotta go, I got a homework emergency.

Well, I did do this last night, it took a little bit and it worked. Hopefully we'll have the same, uh, it will actually work.

I wonder if it's because your Tripoli machine is not getting access to the internet.

Oh, this thing's not on yet. We're trying to fire up the VM. It's probably things in the VM, but I'm expecting a connection call because it's running the DHCP service, so it might actually— the network config rule. Um, but don't worry, it'll— I think it'll eventually come up. It took me a little bit of last night to come up too. But if not, I can do it in a different way. Well, this is just the AOV tool, so while we're waiting, I did replicate instead of this one.

I just did Minecraft, so carry looking at what it would be like. Don't know so much logic.

I had to be like Aha, machine booted and ready. This took forever. See you later. While we're waiting, I'm going to show you what this is like. I fire up my triple-E PC. In this particular case, you press the Escape key, it gives you the boot menu. You can see at the very bottom it says Network Athros Boot Agent, and that's what's actually using the PXE. So I can go down, press that, Enter, and it will say— it says, uh, for the Ethernet controller, and you can see it's getting trying to get a DHCP address. But at this particular point in time, this one's not ready, I believe, to actually start sending out the actual leases yet. But in just a moment, whenever this thing comes up, you will be able to get an address. What will then happen is the address will have a field in the DHCP message that says, hey, there's a PXE boot file. So load the bootloader, and then the configuration files, PXE, the bootloader will actually say, oh well, grab Linux from my virtual machine IP address. I just didn't mount it from NFS. So you don't need that for every PXE boot, but if you're gonna do multi-boot, you need a way to distribute it, and NFS is the way to do it. Let's just off this share. The PXE bootloader will just mount a share from my system and then load the system, actually pull over files. What year are you in? I'm not a student. I work on NCSA on the security operations team. Oh, what is NCSA? Acronyms, I don't know what they are. National Center for Supercomputing Applications. Oh, okay. So, yeah, it's a big academic research organization. It does a lot of high-performance computing. We have one of the world's fastest supercomputers and stuff. It's a cool place to work. Sure. But, so the way this is configured is, while we're waiting, let's go back to the actual, the other one I have, the multiboot. And you can see that this file here, our actual helper functions, This is the menu that's installed at this, these directories and this default. This is the default boot menu in Pixie. And it says, hey, load this menu, it's a .c32 file, set timeout, and then include my Pixie file, pixie.com. So these are files that you can actually set up to set options in the menu. And the first thing it gets is a boot local, so in the menu you can be like, oh, I just want to boot from a regular hard disk. That's one option you can do. The other ones R for each Linux distribution that you want to add it. And then it actually loads the pixie.com file, which just sets color schemes and the width of the terminal, the rows and all that. So nothing too fancy there. And at the end of the script, when it provisions it, after it starts the DHCP, it starts the TFTP daemon, 'cause you actually need something that's unauthenticated, and TFTP is an unauthenticated protocol. And then we'll use that to actually pull the Pixie image and then patch Patches for Kickstart, I don't have all that working yet, but the NFS is to share all the Linux distributions. Let's see, actually this may not come with this. This is actually the slowest server here. Let's try to do your end SSH. Strange. Oh wait, that might not be the right machine. Hold on. Configure it. ssh-config. I have multiple VMs running, so. This is port 22. Yeah, that's right. That should be it. Because it automatically does registration. That's right. So I am in the machine. And so it's becoming up. So let's actually go ahead and make sure that the DHCP service is running. It is not running. That's where we get the address. Start it. Nope, it is— TCP, TCP. Let's first check the, uh, we need to get— we need to have an IP address. So config— oops, let's do this, set to that. And now let's run, see if this will all Yep, now it's starting because I don't have an IP address. Because the way this would work is if you, once you build the VM, you haven't plugged it in the network, my scripts get the IP address of the VM and automatically configure it for you. But in this case, since we're not plugged into the network, we're plugged directly into this machine with no DHCP settings, then I have to have my own. So now that's up and running. So now what we do is we'll go ahead and restart this machine. Escape. It's getting ready to load the Athora's boot agent for PXE. Hit network boot. DHCP. Come on. Come on. Yep, there we go. So there's the Linux menu. And you can see I can choose between booting a Hoka hard drive, Ubuntu, CentOS, DBAN, the direct boot Nukes, uh, this, our ISIS image, uh, GParted and other versions of machines. So I'm gonna go ahead and hit CentOS. Yeah. And then hit CentOS 6.6. And you can see at this point it's actually, uh, loading the Linux kernel and then loading the RAM disk image. And in just a moment we're actually gonna boot. We'll have those in memory and we'll be able to actually mount the root file system. There we go, so it's actually booting Linux right now. You can see the logs coming across the screen. Takes a little bit because it's over the network. So now it's unpacking the root file system image into our ASRAN, uh, initramfs. Switching the clock, so it's actually booting the system. Soon enough we'll actually have, um, the actual install menu of CentOS. So now I just adjust the display. There we go, right there, we can install CentOS. Take a look at that. Pretty badass, if I don't say myself. Took me forever to do all these things because the problem is every image is different. I had to figure out where all the kernels were in each distribution and they all put them in different places between Fedora, CentOS, Ubuntu, Debian, all of them. And then I had to figure out where all the RAM disks were and then copy them to the appropriate places. And I wrote a script to do majority of that. But I can add your distribution as you request. So that's it. And if you want to have your own, at this point, presuming that you actually have a supported one by my scripts, what you can do is go back here. If you go to the Vagrantfile, it actually shows you what you can run. So if you want to create a new ISO image and you want operating system Ubuntu and you want version 14.04, fill this out. This is the way to track it. And then just specify as the URL the ISO image and it will download it and it will unpack it and it will create the appropriate PXE menu for you automatically. So in this particular case, I just commented out the example so people, if they want to look at the file, they can see the examples. But if I ran this in here and we wanted to do, so new ISO, right, and we want to say, hey, let's do Yeah, new version 7 of CentOS. Here's the ISO image and there's my hi message from Capsule, the helper function I talked about earlier. So I check the directory, it mounted the ISO file as read-only, now it's copying it over and now at this particular point we already have a boot configuration with PXE for this version of CentOS. Super simple. Pretty sweet. All right. Again, okay, I don't know if it should be recorded or Nope, it is still being recorded. Great. Alright, so I can cut that out. Alright, so at this particular point, so now we stopped it, we're good again. So we want to go to tasks, this directory we just created. We're going to type vagrant init and that just creates a new, brand new Vagrantfile. It says Vagrantfile has been placed in this directory. So now you're ready to create your first virtual environment. So you just open it up and it's very simple. All the comments, you can comment out the stuff you need. But A base box, you need a box that's built off of. So vagrant boxes list shows you the boxes you have installed. Whoops, boxes, vagrant box, box list, not boxes, list. So you can see I have this many different boxes installed. One for Arch Linux, CentOS, Debian, Fedora, etc. And so that means I don't actually have to go, if someone's using this box already, they don't have to go out to the internet and download it. So let's go back to the Vagrant website. So you go ahead and download this, it's a really sweet tool. And you have packages for a number of operating systems. And what you do is you just create this Vagrantfile to describe what the thing should look like. So we can actually go ahead and first, right off the bat, use a box that I already have. So we saw that one of the boxes was Ubuntu, right? So just a quick look again, Vagrant box list, you see that I have Ubuntu already. Oh, I have it named as Ubuntu 12.04. So I'm actually gonna have to do something to edit that just to match that name, 12.04. And then what happens when I do vagrant up, it's actually gonna import that box in this new directory so that I have Ubuntu 12.04 ready to go. And Vagrant has a certain configuration. It already has the SSH keys ready to go. They're the same. They're shared with all the boxes. The specific format if you wanna create your own boxes and have them shared, but you can go out on the internet and you can download your own boxes. Google Vagrant Boxes. Here we go. And has a tool for discovering boxes. You can actually pull it from their public Vagrant catalog. And so people have their own. So if you want to download Ubuntu Trusty, you can use that. So you can do vagrant box add, you'll have Ubuntu Trusty, right? So just a way to quickly grab the boxes people already have out there. Also, What the unique thing about this is it's just your VM. So you can have all your tools ready to go in this VM and repackage it as a new box or you can write a script, which is what I do, or use some provisioning tool to actually install everything that you want. So we have a brand new machine up running, right? Everything is set, so now we can actually SSH in. So we use Vagrant SSH. We do it in the directory. So everything in Vagrant is based on a directory. So if you have 4 VMs, each VM would have its own directory with its own Vagrant file in each VM. You can do fancy things with Vagrant files. They are Ruby. You can do a loop in there and bring up 4 VMs at once or something like that if you wanted to. I actually have a cluster I've done in Vagrant using Bro. But there it is. So the VM's right there now. I just logged in. But what I'm gonna do is I'm gonna destroy the box. I don't want this box anymore. It's just a test system. So let's destroy it. So it actually removes the VM. And by default, Vagrant works with VirtualBox only. So VagrantBox is the idea of providers. And you can do VirtualBox, you can do VMware. Actually, VMware costs extra money. You have to have to pay a fee for it. But I use VirtualBox for all because it's free and it works well. So if you have to install VirtualBox, then you have to install the Vagrant and then it works. From there, let's go back here. We're going to remove that directory. There'll be some other stuff in here that Vagrant created. So remove that. And what we're actually going to do is we're going to go to my other Vagrant repository directory on GitHub. It'll bring up a system. I will look at some Vagrantfiles actually first. Let's look at this Bro cluster. So here's the Vagrantfile for the Bro cluster. It brings up 3 machines. Actually, I want to do that. Bam. It brings up 3 machines. Here's the first one. Vagrant configure doom. It actually pulls in the trusty box if it's there. So it tries to match it like I showed you with the box list. But if it's not there, pull it from this URL. Ubuntu, for example, hosts their own Vagrant boxes online. So you can find out where they are. And just put in the URL. And if you do not have it on the machine, it'll actually go out and download it and call it trusty. And then next time you need it, it'll import it right away because you already have it. Then we do a loop here. What I'll do, or this manager, what we do is we do, we set the hostname of the virtual machine, set up the networking, give it an IP address, copy over some files. That's the provisioning stuff I was talking about. And then you can tell it to run a script. So you can run a shell script, for example. So it'll run provision-manager.sh. On the system as soon as it comes up. We'll give an example of a simpler file though. That's not the best one. Let's do islet. This brings up an islet system. And— go back here. Okay. So what's actually happening here is it's going to bring up a system called islet, assuming the name of the VM. And then it's going to allocate 1 gigabyte of memory and then 1 CPU for the virtual machine from the host. And you can do other things like sync a folder. You can mount a folder on the host system inside the VM if you wanted to do this, just one line right here. Give the source and destination. You can also do forwarding of guest ports so you can communicate with the host directly, which is nice if you're bringing up maybe a web server stack or LAMP or something with Apache and you want to do some programming or mess with the website. You can do that and you can use your browser like Firefox and just type in the actual guest host or guest port and it will actually connect to that. That's one way to do it. So in this particular case, we're going to bring up this IaaS system. Presumably, I don't have it already running. Ooh, just need to be in the directory. So like I said, it's all directory-based, so you got to be in the directory of each one. I'll check the status here. That's off. We're going to provision a new one, though. We want something that I haven't done yet, at least in a long time. Let's try this one. Not created. That's what we wanted to see. We wanted to see that the VM was not created. So that actually from this process of scratch. So you can see in this one we have a Vagrantfile. We then have some configuration files that were in a script. The script installs all the packages that I want to have after the VM comes up. So this installs cowsay, git, buildessential, shac install, automake, a bunch of development tools, right? And then what actually happens is if you look at the Vagrantfile, these file provisioner types copies a file from the source directory, so in my current directory, to a file on the system. So it's gonna quickly copy over files. It's gonna copy all these configuration files right here without the permission and to this virtual machine. Go ahead and bring it up. So this is also a really cool way, like people actually build these, all these VMs already. If you need some piece of infrastructure, like a LAMP stack, you can just go out to GitHub and find someone's Vagrantfile that's already done and make a new folder called, you make the LAMP stack, cd in that directory, and assuming you have VirtualBox and Vagrant installed and you have that Vagrantfile from that person, type vagrant up just like I did right here, and it's going to build the whole thing from scratch and you can have it on your laptop so you can can play with this stuff. So you don't actually have to spend all the time putting your tools together. A lot of people have done this already. Just to give an example of that, we can go to github.com and we can just type in Vagrant. And here's a bunch of people's stuff. Some with React, some with PageKit, LXC provider for Vagrant, CoreOS. Let's type in— I don't know if you do any database stuff, MySQL maybe. So this is a dead simple LAMP stack, Linux, Apache, MySQL, and PHP. So you literally just go here, you clone this folder, you enter the directory, and you type Vagrant up and you'll have all those tools ready to go to play with. Okay, and it actually installed some stuff and it failed on some stuff. I gotta fix that. I must have wrote my script sometime But at this particular point, the VM is up and I can connect into it and do stuff. But of course, my— I screwed up my script apparently because it failed. Git— oh, Git's not— Git command not found. Okay, so hold on. I'm going to go take this far. I'll give you a good example. Provision— oh, Git is there. Why did that not load? So— package check or package check packages, number of packages. So it just pops in as that, checks, then it should do dpkg.

It'll install a few packages.

That's weird. I should have checked out everything. We can run it again. So if you want to run just the provisioner. So Vega has got a huge menu. You can resume your VMs. You can stop them. You can provision them. You can do whatever. You can run other plugins. You can connect to someone's VM. You can actually do it in a way that you can share VMs. So if you're running a VM in your house, and you want to share it with your coworker who's across the world, you can use Vagrant Connect and actually connect into his virtual machine. You can work on it together, the problem that you want to solve. It's real simple. It does it pretty much automatically. But what is wrong with this one? I don't know. It's, Vagrant provision is how we can actually run the provision script again. It's probably going to fail though because I failed the first one. I should have just worked. So it's going to run it again. Copy over those files at the top. You can see that I have the Vagrantfile. 4 of them, and then run that shell provisioner script I created. Yeah, so yum command not found. Yeah, it's not a system for that. So why is— so something's wrong with the variable being checked out. So we need to do provision, and we need to do— let's check that out. You know, that's the first thing I can think of. And that— what is that? Yeah, that's OS is Debian. So we echo OS. That says Debian because that file existed. And clearly the Red Hat file should not be there. Yeah. So errors say— So you can actually copy the script over as the script, as this file here. So line 69. Oh, I'm an idiot. They're backwards. I messed that up and didn't realize it. Okay, so now. Does anybody see what I did there? Whoops, wrong package managers for the wrong systems. I need to go update all my repos because I copied some of the data. Alright, fair enough. So now we just run vagaprovision. And now, presuming I didn't make any more silly mistakes, it should install everything I wanted. So essentially, I put in the work before time to actually get the stuff up and running. But once you have it ready to go, you can reuse it as much as you want. Oh, I gotta get the new file over. So, um, That's right, let's just start from scratch. Easiest way at this point because it actually has the old version of the script. I would have to actually manually edit it or not, but let's start again. While we're waiting So other commands, Vagrant status tells you if it's up, down, or what it's doing. The machine's powered off at this particular point but ready to come back up. What else do we have? We have Vagrant global status, which is a fairly new one. It actually tells you all the Vagrant machines running on your system. It used to be a pain in the ass. You have to go to every Vagrant folder and run Vagrant status. But now you can see all of them. I have like 4 or 5 VMs running now. That's a cool thing to do. And then the Box stuff. So the Box, how you can add your own boxes and repackage them. So if I go in and I make some changes to a VM, I can use— once I'm in that directory, I've installed the packages I want and everything, I get out of it, I can do vagabox-repackage and it actually changes— takes that VM and makes it a new one with your stuff in it. And then you can pass that around. Apparently I didn't fix it. Yeah, well, I'll look at it later. I don't want to waste your time on that, but that was one problem.