Brian Kernighan: UNIX, C, AWK, AMPL, and Go Programming | Lex Fridman Podcast #109

The following is a conversation with Brian Kernighan, a professor of Computer science at Princeton University. He was a key figure in the computer science community in the early Unix days alongside Unix creators Ken Thompson and Dennis Ritchie. He co authored the C programming language with Dennis Ritchie, the creator of C, and has written a lot of books on programming, computers and life, including the Practice of Programming the GO programming language and his latest Unix A History and a Memoir. He co created awk, the text processing language used by Linux folks like myself. He co designed Ample, an algebraic modeling language that I personally love and have used a lot in my life for large scale optimization. I think I can keep going for a long time with his creations and accomplishments, which is funny because given all that, he's one of the most humble and kind people I've spoken to on this podcast. Quick summary of the ads Two new sponsors, the amazing Self Cooling, eight Sleep Mattress and Raycon Earbuds. Please consider supporting the podcast by going to eightsleep.com lex and going to buy a raycon.com lex click the links.

Buy the stuff.

It really is the best way to support this podcast and the journey I'm on. If you enjoy this thing, subscribe on YouTube, review it with five stars on Apple Podcast, support it on Patreon, or connect with me on Twitter lexfriedman. As usual, I'll do a few minutes of ads now and never any ads in the middle that can break the flow of the conversation. This show is sponsored by Eight Sleep and its incredible Pod Pro mattress that you can check out at eightsleep.com lex to get $200 off the mattress controls temperature with an app and can cool down to as low as 55 degrees. Research shows that temperature has a big impact on the quality of our sleep. Anecdotally has been a game changer for me. I love it. The Pod Pro is packed with sensors that track heart rate, heart rate, variability and and respiratory rate, showing it all on their app once you wake up. Plus if you have a partner, you can control the temperature of each side of the bed. I don't happen to have one, but the Asleep app reminds me that I should probably get on that. So ladies, if a temperature controlled mattress isn't a good reason to apply, I don't know what is. The app's health metrics are amazing, but the cooling alone is honestly worth the money. As some of you know, I don't always sleep, but when I do I choose the Asleep Pod Pro mattress. Check it out@ asleep.com lex to get $200 off. This show is also sponsored by Raycon Earbuds. Get them@buyraycon.com lex they've quickly become my main method of listening to podcasts, audiobooks and music. When I run, do the push ups and pull ups that I've begun to hate at this point or just living life. In fact, I often listen to Brown Noise with ease when I'm thinking deeply about something. It helps me focus the mind. They're super comfortable pair, easily great sound, great bass, six hours of playtime. In fact, for fun, I have one of the earbuds in now and I'm listening to Europa by Santana. Probably one of my favorite guitar songs. It kind of makes me feel like I'm in a music video. So they told me to say that a bunch of celebrities use these like Snoop Dogg, Melissa Etheridge, and Cardi B. I don't even know who Cardi B is, but her earbud game is on point to mention celebrities I actually care about. I'm sure if Richard Feynman was still with us, he'd be listening to the Joe Rogan Experience with with Raycon earbuds. Get them@buyraycon.com Lex it's how they know I sent you and increases the chance that it'll support this podcast in the future. So for all of the sponsors, click all of the links. It really helps this podcast. And now here's my conversation with Brian Kernighan.

UNIX started being developed 50 years ago. Maybe more than 50 years ago. Can you tell the story like you describe in your new book of how Unix was created?

If I can remember that far back, it was some while ago. So I think the gist of it is that at bell Labs in 1969 there were a group of people who had just finished working on the Multics project, which was itself a follow on to ctss. So we can go back sort of an infinite regress in time. But the CTSS was a very, very, very nice timesharing system. It was very nice. I actually used it as that summer I spent in Cambridge in 1966.

What was the hardware there? So what's the operating system? What's the hardware there? What's the CTS look like?

So CTSS looked like kind of like a standard time sharing system. Certainly at the time it was the only timesharing of note.

Let's go back to the basic what's a timesharing system?

Okay. In the beginning was the word and.

The word and then there was timesharing systems.

Yeah, if we go back into, let's call it the 1950s and early 1960s. Most computing was done on very big computers, physically big, although not terribly powerful by today's standards that were maintained in very large rooms. And you use things like punch cards to write your programs on and talk to them. So you would take a deck of cards, write your program on it, send it over a counter, hand it to an operator, and some while later, back would come something that said, oh, you made a mistake, and then you'd recycle. And so it's very, very slow. So the idea of timesharing was that you take basically that same computer, but connect to it with something that looked like an electric typewriter that could be a long distance away, it could be close. But fundamentally, what the operating system did was to give each person who was connected to it and wanting to do something a small slice of time on to do a particular job. So I might be editing a file, so I would be typing, and every time I hit a keystroke, the operating system would wake up and said, oh, he typed character. Let me remember that. Then it go back to doing something else. It would be going around and around a group of people who were trying to get something done, giving each a small slice of time and giving them each the illusion that they pretty much had the whole machine to themselves, and hence time sharing. That is, sharing the computing time resource of the computer among a number of.

People who are doing it without the individual people being aware that there's others. In a sense, the illusion, the feeling, is that the machine is your own.

Pretty much, that was the idea. Yes. You had. If it were well done and if it were fast enough and other people weren't doing too much, you did have the illusion that you had the whole machine to yourself, and it was very much better than the punch card model. And so ctss, the compatible timesharing system, was, I think, arguably the first of these. It was done, I guess, technically in 64 or something like that. It ran on an IBM 7094, slightly modified to have twice as much memory as the norm. It had two banks of 32k words instead of one. So.

32K words?

Yeah, each word was 36 bits. So call it, you know, about 150 kilobytes times two. So by today's standards, that's down in the noise. But at the time, that was a lot of memory, and memory was expensive. So CTSS was just a wonderful environment to work on. It was done by the people at mit, led by Fernando Corbetov. Corby, who died just earlier this year and a bunch of other folks then. So I spent the summer of 66 working on that, had a great time, met a lot of really nice people, and indirectly knew of people at Bell Labs who were also working on a follow on to CTSS that was called multics. So multics was meant to be the system that would do everything that CTSS did, but do it better for a larger population. All the usual stuff.

Now the actual time sharing, the scheduling, what's the algorithm that performs the scheduling? What's that look like? How much magic is there? What are the metrics? How does it all work in the beginning?

So the answer is I don't have a clue. I think the basic idea was nothing more than who all wants to get something done. Suppose that things are very quiet in the middle of the night. Then I get all the time that I want. Suppose that you and I are contending at high noon for something like that. Then probably the simplest algorithm is a round robin, one that gives you a bit of time. Gives me a bit of time. And then we could adapt to that. Like what are you trying to do? Are you text editing or are you compiling or something? And we might adjust the scheduler according to things like that.

So, okay, so multics was trying to just do some of the. Clean it up a little bit.

Well, it was, it was meant to be much more than that. So multics was the multiplexed information and computing service and it was meant to be a very large thing that would provide computing utility, something that, where you could actually think of it as just a plug in the wall service, sort of like cloud computing today. Same idea, but 50 odd years earlier. And so what multics offered was a richer operating system environment, a piece of hardware that was better designed for doing the kind of sharing of resources and presumably lots of other things.

Do you think people at that time had the dream of what cloud computing is starting to become now? Which is computing is everywhere that you can just plug in almost. You never know how the magic works. You just kind of plug in at your little computation that you need to perform and it does it. Was that the dream?

I don't know where that was the dream. I wasn't part of it at that point. Remember I was an intern for Summer. But my senses, given that it was over 50 years ago. Yeah, they had that idea that it was an information utility, that it was something where if you had a computing task to do, you could just go and do it. Now I'm betting That they didn't have the same view of computing for the masses. Let's call it the idea that, you know, your grandmother would be shopping on Amazon. I don't think that was part of it. But if your grandmother were a programmer, it might be very easy for her to go and use this kind of utility.

What was your dream of computers at that time? What did you see as the future of computers? Could you have predicted what computers are today in a sense?

Oh, short answer, Absolutely not. I have no clue. I'm not sure. I had a dream. It was a dream job in the sense that I really enjoyed what I was doing. I was surrounded by really, really nice people. Cambridge is a very fine city to live in in the summer, less so in the winter when it snows. But in the summer it was a delightful time. And so I really enjoyed all of that stuff. And I learned things and I think the good fortune of being there for summer led me then to get a summer job at Bell Labs the following summer. And that was going to be useful for the future.

So this Bell Labs is this magical, legendary place. So first of all, where is Bell Labs? And can you start talking about that journey towards Unix at Bell Labs?

Yeah. So Bell Labs is physically scattered around at the time, scattered around New Jersey. The primary location was in a town called Murray Hill, or a location called Murray Hill. It was actually across the boundary between two small towns in New Jersey called New Providence and Berkeley Heights. Think of it as about 15, 20 miles straight west of New York City and therefore about an hour north of here in Princeton. And at that time it had make up a number three or four thousand people there, many of whom had PhDs and mostly doing physical sciences, chemistry, physics, materials, kinds of things, but very strong math and rapidly growing interest in computing as people realized you could do things with computers that you might not have been able to do before. You could replace labs with computers that had worked on models of what was going on. So that was the essence of Bell Labs. And again, I wasn't a permanent employee there. That was another internship. I got lucky in internships.

I mean, if you could just linger on it a little bit, what was in the air there because some of the number of Nobel Prizes, the number of touring awards, and just legendary computer scientists that come from their inventions, including developments, including Unix, is just. Is unbelievable. So was there something special about that place?

Oh, I think there was very definitely something special. I mentioned the number of people, so very large number of people, very highly skilled and working in an environment where there was always something interesting to work on because the goal of Bell Labs, which was a small part of AT&T, which provided basically the country's phone service, the goal of AT&T was to provide service for everybody. And, and the goal of Bell Labs was to try and make that service keep getting better. So improving service and that meant doing research on a lot of different things. Physical devices like the transistor or fiber, optical cables or microwave systems, all of these things the Labs worked on. And it was kind of just the beginning of real boom times in computing as well. Because when I was there I went there first in 66. So computing was at that point fairly young. And so people were discovering that you could do lots of things with computers.

So how's Unix born?

So multics, in spite of having an enormous number of really good ideas, lots of good people working on it, fundamentally didn't live up, at least in the short run and I think ultimately really ever to its goal of being this information utility. It was too expensive and certainly what was promised was delivered much too late. And so in roughly the beginning of 1969, Bell Labs pulled out of the project. The project at that point had included mit, Bell Labs and General Electric. General Electric made computers, so General Electric was the hardware operation. So Bell Labs, realizing this wasn't going anywhere on a timescale they cared about, pulled out of the project. And this left several people with an acquired taste for really, really nice computing environments, but no computing environment. And so they started thinking about what could you do if you were going to design a new operating system that would provide the same kind of comfortable computing as CTSS had, but also the facilities of something like multics brought forward. And so they did a lot of paper design stuff. And at the same time Ken Thompson found what is characterized as a little used PDP 7 where he started do experiments with file systems. Just how do you store information on a computer in an efficient way? And then this famous story that his wife went away to California for three weeks, taking their one year old son and three weeks and he sat down and wrote an operating system which ultimately became Unix. So software productivity was good in those days.

So PDP, what's a PDP 7? So it's a piece of hardware?

Yeah, it's a piece of hardware. It was one of early machines made by Digitally Equipment Corporation Deck and it was a minicomputer so called. It had. I would have to look up the numbers exactly, but it had a very small amount of memory, maybe 16k, 16 bit words or something. Like that, relatively slow, probably not super expensive. Maybe again, making this up, I'd have to look it up. $100,000 or something like that, which is.

Not super expensive in those days, right?

It was expensive. It was enough that you and I probably wouldn't be able to buy one. But a modest group of people could get together. But in any case, in it came out, if I recall, in 1964. So by 1969, it was getting a little obsolete, and that's why it was little used.

If you can sort of comment, what do you think it's like to write an operating system like that? So that process that Ken went through in three weeks, because you were. I mean, you're part of that process, you contributed a lot to Unix's early development. So what do you think it takes to do that first step, that first kind of from design to reality on the pdp?

Well, let me correct one thing. I had nothing to do with it, so I did not write it. I have never written operating system code. And so I don't know. Now, an operating system is simply code. And this first one wasn't very big, but it's something that lets you run processes of some, let you execute some kind of code that has been written that lets you store information for periods of time so that it doesn't go away when you turn the power off or reboot or something like that. And there's a kind of a core set of tools that are technically not part of an operating system, but you probably need them. In this case, Ken wrote an assembler for the PDP 7 that worked. He needed a text editor so that he could actually create text. He had the file system stuff that he had been working on, and then the rest of it was just a way to load things, executable code from the file system into the memory, give it control, and then recover control when it was finished or in some other way quit.

What was the code written in the. Primarily the programming language. Was it an Assembly.

Yeah, PDP 7 assembler that Ken created. These things were assembly language until probably the. Call it 1973 or 74, something like that.

Forgive me if it's a dumb question, but it feels like a daunting task to write any kind of complex system in assembly.

Absolutely.

It feels like impossible to do any kind of what we think of as software engineering with assembly, because to work on a big picture sort of, I think it's hard.

It's been a long time since I wrote assembly language. It is absolutely true that in assembly language if you make a mistake, nobody tells you there are no training wheels whatsoever. And so stuff doesn't work. Now what?

There's no debuggers?

Well, there could be debuggers, but that's the same problem, right? How do you actually get something that will help you debug it? So part of it is an ability to see the big picture. Now these systems were not big in the sense that today's pictures. Actually the big picture was in some sense more manageable. I mean, then realistically there's an enormous variation in the capabilities of programmers. And Ken Thompson, who did that first one, is, is kind of the singularity in my experience of programmers. With no disrespect to you or even to me, he's, he's in it several leagues removed.

I know there's levels this, it's, it's a, it's a fascinating thing that there are unique stars in particular in the programming space and at a particular time. You know, the time matters to the timing of when that person comes along and the. A wife does have to leave. Like there's this weird timing that happens that. And then all of a sudden something beautiful is created. I mean, how does it make you feel that there's a system that was created in, in three weeks, or maybe you can even say on a whim, but not really, but of course, quickly that is. Now, you could think of most of the computers in the world run on a UNIX like system, right? How do you inter. Like if you kind of zoom from the alien perspective, if you're just observing Earth and all of a sudden these computers took over the world and they started from this little initial seed of unix. How does that make you feel?

It's quite surprising. And you asked earlier about prediction, the answer is no. There's no way you could predict that kind of evolution. And I don't know whether it was inevitable or just a whole sequence of blind luck. I suspect more of the latter. And so I look at it and think, gee, that's kind of neat. I think the real question is, what does Ken think about that? Because he's the guy arguably from whom it really came. You know, tremendous contributions from Dennis Ritchie and then others around in that Bell Labs environment. But, you know, if you had to pick a single person, that would be Ken.

So you've written a new book, UNIX A History and a Memoir. Are there some memorable human stories, funny or profound from that time? They just kind of stand out.

Oh, there's a lot of them in a sense. And again, it's a question of can you resurrect them in real time. Memory fails. But I think part of it was that Bell Labs at the time was. Was a very special kind of place to work because there were a lot of interesting people and the environment was very, very open and free. It was a very cooperative environment, very friendly environment. And so if you had an interesting problem, you go and talk to somebody and they might help you with the solution. And. And it was a kind of a fun environment too, in which people did strange things and often tweaking the bureaucracy in one way or another.

So rebellious in part, in certain kinds of ways.

In some ways, yeah, absolutely. I think most people didn't take too kindly to the bureaucracy and I'm sure the bureaucracy put up with an enormous amount that they didn't really want to.

So maybe to linger on it a little bit. Do you have a sense of what the philosophy that characterizes Unixes? The design, not just the initial, but just carry through the years, just being there, being around it. What's the fundamental philosophy behind the system?

I think one aspect, the fundamental philosophy was to provide an environment that made it easy to write or easier productive to write programs. So it was meant as a programmer environment. It wasn't meant specifically as something to do some other kind of job. For example, it was used extensively for word processing, but it wasn't designed as a word processing system. It was used extensively for lab control, but it wasn't designed for that. It was used extensively as a front end for big other systems, big dumb systems, but it wasn't designed for that. It was meant to be an environment where it was really easy to write programs that. So the programmers could be highly productive. And part of that was to be a community. And there's some observation from Dennis Ritchie, I think at the end of the book that says that, that from his standpoint, the real goal was to create a community where people could work as programmers on a system. And I think in that sense, certainly for many, many years, it succeeded quite well at that. And part of that is the technical aspects of. Because it made it really easy to write programs. People did write interesting programs, those programs tended to be used by other programmers. And so it was kind of a virtuous circle of more and more stuff coming out that was really good for.

Programmers and you were part of that community of programmers. So what was it like writing programs on that early Unix?

It was a blast. It really was. You know, I like to program, I'm not a terribly good programmer, but it was a lot of fun to write code. And in the early days there was an enormous amount of what you would today, I suppose, call low hanging fruit. People hadn't done things before, and this was this new environment and the whole combination of nice tools and very responsive system and tremendous colleagues made it possible to write code. You could, you could have an idea in the morning, you could do a, you know, an experiment with it. You could have something limping along that night or the next day and people would react to it and they would say, oh, that's wonderful, but you're really screwed up here. And. And the feedback loop was then very, very short and tight. And so a lot of things got developed fairly quickly that in many cases still exist today. And I think that was part of what made it fun, because programming itself is fun. It's puzzle solving in a variety of ways. But I think it's even more fun when you do something that somebody else then uses. Even if they whine about it not working, the fact that they used it is part of the reward mechanism.

And what was the method of interaction, the communication, that feedback loop? I mean, this is before the Internet.

Certainly before the Internet, it was mostly physical right there. You know, somebody would come into your office and say something.

So these places are all close by, like offices are nearby. So really lively interaction.

Yeah, yeah. No, Bell Labs was fundamentally one giant building and most of the people were involved in this unique stuff, were in two or three quarters and there was a room. Oh, how big was it? Probably call it 50ft by 50ft, make up a number of that, which had some access to computers there as well as in offices. And people hung out there and it had a coffee machine. And so there was a. It was mostly very physical. We did use email, of course, and. But it was fundamentally all, for a long time, all on one machine, so there was no need for Internet.

Fascinating to think about what computing would be today without Bell Labs. It seems so many, the people being in the vicinity of each other, sort of getting that quick feedback, working together, so many brilliant people. I don't know where else that could have existed in the world. I've been given how that came together. Yeah. How does that make you feel that that's that little element of history?

Well, I think that's very nice, but in a sense it's survivor bias. And if it hadn't happen at Bell Labs, there were other places that were doing really interesting work as well. Xerox park is perhaps the most obvious one. Xerox park contributed enormous amount of good material. And many of the things we take for granted today in the same way came from Xerox park experience. I don't think they capitalized in the long run as much. Their parent company was perhaps not as lucky in capitalizing on this. Who knows? But that would, that's certainly another place where there was a tremendous amount of influence. There were a lot of good university activities. MIT was obviously no slouch in this kind of thing and others as well.

So Unix turned out to be open source because of the various ways that AT&T operated and sort of had to. It was. The focus was on telephones.

So I think that's a mischaracterization in the sense it absolutely was not open source. It was very definitely proprietary licensed, but. But it was licensed freely to universities in source code form for many years. And because of that, generations of university students and their faculty people grew up knowing about Unix and there was enough expertise in the community that it then became possible for people to kind of go off in their own direction and build something that looked UNIX like. The Berkeley version of Unix started with that licensed code and gradually picked up enough of its own code contributions, notably from people like Bill Joy, that eventually it was able to become completely free of any AT&T code. Now there was an enormous amount of legal jockeying around this, that in the late, early to late 80s, early 90s, something like that. And then I guess the open source movement might have started when Richard Stallman started to think about this in the late 80s and by 1991, when Torvalds decided he was going to do a Unix like operating system, there was enough expertise that in the community that first he had a target, he could see what to do because the kind of the Unix system call interface and the tools and so on were there. And so he was able to build an operating system that at this point, when you say Unix in many cases what you're really thinking is Linux.

Linux, yeah. But it's funny that from my distant perception I felt that Unix was open source without actually knowing it. But what you're really saying it was just freely licensed.

It was freely licensed.

So it felt open source because universities are not trying to make money. So it felt open source in a sense that you can get access if you wanted.

Right? And a very, very, very large number of universities had the license and they were able to talk to all the other universities who had the license. And so technically not open, technically belonging to AT&T, pragmatically pretty open.

And so there's A ripple effect that all the faculty and the students then grew up, and then they went throughout the world and permeate in that kind of way. So what kind of features do you think make for a good operating system? If you take the lessons of Unix, you said make it easy for programmers, that seems to be an important one. But also, Unix turned out to be exceptionally robust and efficient.

Right.

So is that an accident when you focus on the programmer, or is that a natural outcome?

I think part of the reason for efficiency was that it began on extremely modest hardware, Very, very, very tiny. And so you couldn't get carried away. You couldn't do a lot of complicated things because you just didn't have the resources, either processor speed or memory. And so that enforced a certain minimality of mechanisms and maybe a search for generalizations so that you would find one mechanism that served for a lot of different things rather than having lots of different special cases. I think the file system in Unix is a good example of that. The file system interface in its fundamental form is extremely straight. And that means that you can write code very, very effectively for the file system. And then one of those generalizations is that, gee, that file system interface works for all kinds of other things as well. And so in particular, the idea of reading and writing to devices is the same as reading and writing to a disk that has a file system. And then that gets carried further in other parts. The world processes become, in effect, files in a file system. And the Plan 9 operating system, which came along, I guess, in the late 80s or something like that, took a lot of those ideas from the original Unix and tried to push the generalization even further. So that in Plan 9, a lot of different resources, our file systems, they all share that interface. So that would be one example where finding the right model of how to do something means that an awful lot of things become simpler. And it means, therefore, that more people can do useful, interesting things with them without having to think as hard about it.

So you said you're not a very good programmer.

True.

You're the most modest human being. Okay, but you'll continue saying that I understand how this works, but you do radiate a sort of love for programming. So let me ask, do you think programming is more an art or a science? Is it creativity or kind of rigorous?

I think it's some of each. It's some combination. Some of the art is figuring out what it is that you really want to do. What should that program be, what would make a good program? And that's some Understanding of what the task is, what the people who might use this program want. And I think that's art in many respects. The science part is trying to figure out how to do it well. And some of that is real computer sciencey stuff like what algorithm should we use at some point? Mostly in the sense of being careful to use algorithms that will actually work properly, scale properly avoiding quadratic algorithms when a linear algorithm should be the right thing, that kind of more formal view of it. Same thing for data structures, but also it's, I think, an engineering field as well. And engineering is not quite the same as science, because engineering, you're working with constraints, you have to figure out not only so what is a good algorithm for this kind of thing, but what's the most appropriate algorithm given the amount of time we have to compute, the amount of time we have to program, what's likely to happen in the future with maintenance? Who's going to pick this up in the future? All of those kind of things that if you're an engineer, you get to worry about. Whereas if you think of yourself as a scientist, well, you can maybe push them over the horizon in a way. And if you're an artist, what's that?

So just on your own personal level, what's your process like of writing a program? Say, small and large, sort of tinkering with stuff? Do you just start coding right away and just kind of evolve iteratively with a loose notion? Or do you plan on a sheet of paper first and then kind of design in a second in what they teach you in the kind of software engineering courses in undergrad or something like that? What's your process like?

It's certainly much more the informal incremental. First, I don't write big programs at this point. It's been a long time since I wrote a program that was more than, I call it, a few hundred or more lines, something like that. Many of the programs I write are experiments for either something I'm curious about or often for something that I want to talk about in a class. And so those necessarily tend to be relatively small. A lot of the kind of code I write these days tends to be for sort of exploratory data analysis where I've got some collection of data and I want to try and figure out what on earth is going on in it. And for that those programs tend to be very small. Sometimes you're not even programming, you're just using existing tools like counting things, or sometimes you're writing OX scripts because two or three lines will tell you something about a piece of data and then when it gets bigger, well, then I will probably write something in Python because that scales better up to call it a few hundred lines or something like that. And it's been a long time since I wrote programs that were much more than that.

Speaking of data exploration in awk, first, what is awk?

So AWK is a scripting language that was done by myself, Al Aho and Peter Weinberger. We did that originally in the late 70s. It was a language that was meant to make it really easy to do quick and dirty tasks like counting things or selecting interesting information from basically all text files, rearranging it in some way.

Or summarizing it runs a command on each line of a file. I mean, there's. It's still exceptionally widely used today.

Oh, absolutely, yeah.

It's so simple and elegant, sort of the way to explore data. Turns out you can just write a script that does something seemingly trivial in a single line, and that giving you that slice of the data somehow reveals something fundamental about the data. And that seems to work still.

Yeah, it's very good for that kind of thing. And that's sort of what it was meant for. I think what we didn't appreciate was that the model was actually quite good for a lot of data processing kinds of tasks and that it's kept going as long as it has because at this point it's over 40 years old and it's still, I think, a useful tool. And, well, this is paternal interest, I guess. But I think in terms of programming languages, you get the most bang for the buck by learning awk. And it doesn't scale the big programs, but it does pretty, pretty darn well on these little things where you just want to see all the somethings in something. So, yeah, I find I probably write more awkward than anything else at this point.

So what, what kind of stuff do you love about awk? Like, is there, if you can comment on sort of things that give you joy when you can, in a simple program, reveal something about the data? Is there something that stands out from particular features?

I think it's mostly the selection of default behaviors. You sort of hinted at it a moment ago. What OK does is to read through a set of files and then within each file it reads through each of the lines, and then on each of the lines it has a set of patterns that it looks for. That's your AWK program. And if one of the patterns matches, there is a corresponding action that you might perform. And so it's Kind of a quadruply nested loop or something like that. And that's all completely automatic. You don't have to say anything about it, you just write the pattern and the action and then run the data by it. And so that paradigm for programming is very natural and effective one, and I think we captured that reasonably well in awk. And it does other things for free as well. It splits the data into fields so that on each line there's fields separated by white space or something. And so it does that for free, you don't have to say anything about it. And it collects information as it goes along. Like what line are we on? How many fields are there on this line? So lots of things that just make it so that a program, which in another language, let's say python, would be 5, 10, 20 lines in arc is one or two lines.

So because it's one or two lines, you can do it on the shell. Like you don't have to open up another whole thing, you can just do it right there in the interaction with the, with the operative. Is there other shell commands that you love over the years? Like, you really enjoyed using grep?

Grep's the only one. Yeah, GREP does everything.

So GREP is a kind of. What is it? A simpler version of awk, I would say in.

In some, some sense, yeah. Right.

Because what is grep?

So GREP is it basically searches the input for particular patterns, regular expressions technically of a certain class. And it has that same paradigm that AWK does. It's a pattern action thing. It reads through all the files and then all the lines in each file, but it has a single pattern, which is the regular expression you're looking for, and a single action printed if it matches. So in that sense it's a much simpler version. And you could write crap in AWK as a one liner. And I use GREP probably more than anything else at this point, just because it. It's so convenient and natural.

Why do you think it's such a powerful tool, grep Knock. Why do you think operating systems like Windows, for example, don't have it? Sort of. You can, of course, I use the the. Which is amazing. Now there's Windows for Linux, like which you could basically use all the fun stuff like AWK and GREP inside of Windows, but Windows naturally sort of as part of the graphical interface. The simplicity of grep, sort of searching through a bunch of files and just popping up naturally. Why do you think that's unique to the UNIX Linux environment?

I don't know. It's not strictly unique, but it's certainly focused there. And I think some of it's the weight of history that Windows came from. Ms. DOS, Ms. DOS was a pretty pathetic operating system, although common on, you know, unboundedly large number of machines. But somewhere in roughly the 90s, Windows became a graphical system. And I think Microsoft spent a lot of their energy on making that graphical interface what it is. And that's a different model of computing. It's a model computing that where you point and click and sort of experiment with menus. It's a model computing works right rather well for people who are not programmers just want to get something done whereas teaching something like the command line to non programmers turns out to sometimes be an uphill struggle. And so I think Microsoft probably was right in what they did. Now you mentioned Whistle or whatever it's called the Linux Linux.

I wonder what's pronounced WSL is what I've never actually pronounced the Whistle.

I like it, I have no idea. But there have been things like that for long Cygwin for example, which is a wonderful collection of take all your favorite tools from Unix and Linux and just make them work perfectly on Windows. And so that's something that's been going on for at least 20 years, if not longer. And I use that on my one remaining Windows machine routinely because if you're doing something that is batch computing command line suitable for command line, that's the right way to do it because the Windows equivalents are if nothing else, not familiar to me.

But I would definitely recommend to people to if they don't use Sigma to try. Whistle. Yes, I've been so excited that I could use Bash W Bash write scripts quickly in Windows. It's changed my life. Okay, what's your perfect programming setup? What computer? What operating system? What keyboard? What editor?

Yeah, perfect is too strong a word. It's way too strong a word. What I use by default, I have at this point a 13 inch MacBook Air which I use because it's kind of a reasonable balance of the various things I need. I can carry it around, it's got enough computing horsepower. Screen's big enough, the keyboard's okay and so I basically do most of my computing on that. I have a big imac in my office that I use from time to time as well, especially when I need a big screen but otherwise it tends not to be used as much. Editor I use mostly sam, which is an editor that Rob pike wrote long ago at Bell Labs did that.

Sorry to interrupt. Does that precede Vice post it post.

Dates both VI and Emacs. It is derived from Rob's experience with ED and vi. That's the original UNIX editor.

Oh, wow.

Dated probably before you were born.

So what's actually, what's the history of editors? Can you briefly. Because it's such a fat. I used Emacs, I'm sorry to say. So sorry to come out with that, but what's, what's the kind of interplay there?

So in ancient, ancient times, like call it the first time sharing systems, going back to what we were talking about, there were editors. There was an editor on CTSS that I don't even remember what it was called. It might have been Edit, where you could type text, program text and it would do something or other document text.

You could save, you could edit it.

You know, the usual thing that you would get in an editor. And Ken Thompson wrote an editor called qed which was very, very powerful. But these were all totally a command based. They were not mouse or cursor based because it was before mice and even before cursors because they were running on terminals that printed on paper. Okay. No, no CRT type displays, let alone LEDs. And so then when Unix came along, Ken took QED and stripped it way, way, way, way down. And that became an editor that he called ed. It was very simple, but it was a line oriented editor. And so you, you could load a file and then you could talk about the lines one through the last line and you could, you know, print ranges of lines, you could add text, you could delete text, you could change text, or you could do a substitute command that would change things within a line or within groups of lines.

So you can work on a. Parts of a file essentially?

Yeah, you could work on any part of it, the whole thing or whatever, but it was entirely command line based and it was entirely on paper. Paper. And that meant that you changed. Yeah, right. Real paper. And so if you change the line, you had to print that line, using up another line of paper to see what the change caused. Okay. Yeah. So when, when CRT displays came along. Yeah, Then you could start to use cursor control and you could sort of move where you were on the screen.

In without reprinting, without printing.

And one of. There were a number of editors there. The one that I was most familiar with and still use is vi, which was done by Bill Joy. And so that dates from probably the late 70s as a guess. And it took it full advantage of the cursor controls. I suspect that Emacs Was roughly at the same time. But I don't know. I've never internalized Emacs. So I use. At this point, I stopped using ed, although I still can. I use VI sometimes and I use SAM when I can.

And SAM is available on most systems.

It was. It is available. You have to download it yourself from typically the Plan 9 operating system distribution. It's been maintained by people there.

And so I'll get home tonight. I'll try it. It's cool. It sounds. It's sounds. Sounds fascinating. Although my love is with Lisp and Emacs, I've went into that hippie world of.

I think it's a lot of things. What religion were you brought up with?

Yeah, that's true, that's true. Most of the actual programming I do is C, C and Python. But my weird sort of. Yeah, my religious upbringing is in Lisp. So can you take on the impossible task and give a brief history of programming languages from your perspective?

So I guess you could say programming languages started probably in what, the late 40s or something like that. People used to program computers by basically putting in zeros and ones, using something like switches on a console or maybe holes in paper tapes, something like that. So extremely tedious. Awful, whatever. And so I think the first programming languages were relatively crude assembly languages where people would basically write a program that would convert mnemonics like add. Add into whatever the bit pattern was that corresponded to an add instruction. And they would do the clerical work of figuring out where things were. So you could put a name on a location in a program and the assembler would figure out where that corresponded to when the thing was all put together and dropped into memory. And early on, and this would be the late 40s and very early 50s, there were assemblers written for the various machines that people used. You may have seen in the paper. Just a couple days ago, Tony Burker died. He did this thing in Manchester called the. Called autocode. A language which I knew only by name, but it sounds like it was a flavor of assembly language sort of a little higher in some ways. And it replaced a language that Alan Turing wrote, which you put in zeros and ones, but you put them in backwards order because that was hardware worked very straight.

That's right. Yeah. Yeah, that's right. It's backwards.

So assembly languages, then. Let's call that the early 1950s. And so every different flavor of computer has its own assembly language. So the EDSAC had hits and the Manchester had hits, and the IBM whatever, 7890 or 704 or whatever had hit and so on. So everybody had their own assembly language.

And assembly languages have a few commands, addition, subtraction, then branching of some kind, if then type of situation.

Right. They have exactly, in their simplest form, at least one instruction per, or one assembly language instruction per instruction in the machine's repertoire. And so you have to know the machine intimately to be able to write programs in it. And if you write an assembly language program for one kind of machine and then you say, geez, it's nice, I'd like it in a different machine. Start over. Okay, so very bad. And so what happened in the late 50s was people realized you could play this game again and you could move up a level in writing or creating languages that were closer to the way that real people might think about how to write code. And there were, I guess arguably three or four at that time period. There was fortran, which came from IBM, which was formula translation, meant to make it easy to do scientific and engineering computations.

I didn't know that. Formula translation, that's.

Wow, look at Farm. There was cobol, which is the common business oriented language that Grace Hopper and others worked on, which was aimed at business kinds of tasks. There was Algol, which was mostly meant to describe algorithmic computations. I guess you could argue BASIC was in there somewhere. I think it's just a little later. And so all of those moved the level up. And so they were closer to what you and I might think of as we were trying to write a program and they were focused on different domains. FORTRAN for formula translation, engineering computations, let's say COBOL for business, that kind of thing.

And still used today, at least FORTRAN probably.

Oh yeah, COBOL too. But the deal was that once you moved up that level, then you, let's call it fortran. You had a language that was not tied to a particular kind of hardware because a different compiler would compile for different kind of hardware. And that meant two things. It meant you only had to write the program once, which is very important. And it meant that you could, in fact, if you were a random engineer, physicist, whatever, you could write that program yourself. You didn't have to hire a programmer to do it for you might not be as good as you'd get with a real programmer, but it was pretty good. And so it democratized and made much more broadly available the ability to write code.

So it puts the power of programming into the hands of people like you.

Yeah, anybody who wants to, who is willing to invest some time in learning a programming language and is not then tied to a particular kind of computer. And then in the 70s you get system programming languages of which C is the survivor.

And what, what, what does system programming languages mean?

Programs that programming languages that would take on the kinds of things that would necessary to write so called system programs. Things like text editors or assemblers or compilers or operating systems themselves, those kinds of things.

And Fortran, feature rich, they have to be able to do a lot of stuff, a lot of memory management, access processes and all that kind of stuff. Processing.

It's a different flavor of what they're doing. They're much more in touch with the actual machine, but in a positive way. That is you can talk about memory in a more controlled way. You can talk about the different data types that the machine supports and underway and more ways to structure and organize data. And so the system programming languages, there was a lot of effort in that in the call it the late 60s, early 70s. C is I think the only real survivor of that. And then what happens after that?