Brian Kernighan: UNIX, C, AWK, AMPL, Go, Bell Labs, etc

Software
, , , ,
1

Interesting discussion with Brian Kernighan:

Brian expresses in a humble way some of the experiences he has been through, and some of the history behind the great events in computing (UNIX, C, etc).

At 20:12, Brian discusses the environment of Bell Labs:

Bell Labs at the time was very special kind of place to work, because there were a lot of interesting people, and the environment was very, very open and free – was a very cooperative environment, was a very friendly environment, so if you had an interesting problem, you’d go and talk to somebody and they might help you with the solution.

I recall him talking more about this environment elsewhere in the discussion, but can’t find the location offhand. This type of environment captures the essence of “community” which somehow was created in Bell Labs before OSS and distributed collaboration was possible.

Around 31:24, he discusses programming – is it art or science. Some of his thoughts:

  • art: figuring out what to do
  • science: how to do it well
  • engineering: working with constraints – not only what is a good algorithm, but what is the most appropriate algorithm given the amount of time we have to compute, the amount of time we have to program, what’s likely to happen in the future with maintenance, who’s going to pick this up in the future – all those kinds of things if you’re an engineer you get to worry about, whereas if you think of yourself as a scientist, maybe you can push those over the horizon in a way, and if you’re an artist – “what’s that?”

There are a lot of good ideas. There is a lot of interesting technology. But one of the keys to product development is figuring out what technology to use and when.

2

Another good discussion with Brian:

They discuss how working on a limited, obsolete PDP machine at the time somewhat forced them to do things efficiently and simply. One of the results is that in Unix, everything (even devices) are a file. There might be a parallel to Simple IoT where everything in the system is a node.

3

Some more notes on the Changelog interview with Brian …

They discuss if there are some parallels between the success of Unix and the Web:

Brian Kernighan Says

Brian Kernighan

Yeah, that’s a really interesting question. I think you’re right, I see some parallels that might even be instructive. I mean, fundamentally, it’s the core simplicity of the thing. These are not complicated; they are simple. As we’ve talked about earlier, the essence of Unix is a handful of ideas that work really well together. I mean, the hierarchical filesystem, the programmable shell redirection, not too many system calls… And interestingly, text is kind of the universal medium of exchange of information.

Now you look at the web. There’s (if you wanna call them) system-calls-y kinds of things. There’s HTTP, HTML, the URL. And that’s it. There is nothing else. It’s got the internet as an infrastructure… Oh, and everything that goes across the web is text. So that commonality there I think is quite real. And you know, Berners-Lee created that stuff kind of out of nothing, but building on what was already there. So he had a very clean, simple idea of what to do.

HTML is basically a dialect of SGML, which derives from GML, and on and on into the past, but he simplified it and cleaned it up in a way that made it very useful for this kind of application. So I think there’s actually quite a bit of parallel there.

Brian also has some opinions on the difference between a computer and modern mobile devices:

Brian Kernighan Says

Brian Kernighan

Yeah, the iPad is a nice example of that, in a way. I mean, I have an iPad; I turn it on maybe once every six months or something like that, because it’s an utterly useless device, because it only lets me do what Apple thought I wanted to do. And I don’t know how to make it do most of the things, because it requires funny artificial “wiggle your fingers while rubbing your elbow with something else” to make something happen, which with a command line interface - and this is the old-fogey speaking - I could type two or three characters and I’d be done. But I don’t use it primarily because it’s not programmable. It is a totally useless device to me. And a phone I think falls into that same category. I use a phone occasionally; I haven’t turned mine on for several days, but you know… I can’t program it either, so not as interesting or fun.

They also discuss the constraints of the obsolete PDP-7 that Unix was originally developed on and how that may have contributed to the simplicity and efficiency of Unix.

Several times the question is brought up – what can we learn from history? Brian has written a book on the history of Unix – now one of the next books on my list.

4

yes good observations. For a developer/engineer’s point of view locked devices are a put off for sure.

5

I found this a good talk from Brian presented at linux.conf.au 2022, a good jog down the history to know
where we stand now and how it all came together.

7

Finished Brian’s book: UNIX, A History and a Memoir – excellent book – highly recommend! Bell Labs was an exceptional place and the world is different because of it. We can learn a lot from the culture. The ideas of openness, collaboration, freedom, etc were all ahead of its time. Today, some of this is likely manifest in open source projects today. However, the distraction factor is also much higher. The environment of Bell Labs where you could have collaboration and focus must have been amazing.

Some quotes that stood out to me (again, get the book and read it to get the full context):

  • This also led to our first rule of programming style: Write clearly – don’t be too clever.
  • He (Dick Hamming) used to say that Friday afternoons were for thinking great thoughts, so he sat back and thought, though he was always welcoming to visitors like me at any time.
  • By contrast (to university rankings which favored people with deep expertise in narrow fields), Bell Labs created a rand order for every researcher, from the bottom up. Each department head ranked his or her people; those rankings were merged by department heads within a center, and those in turn by the next two levels, so by the end everyone’s approximate position in the whole population was determined. A person who did great work in a narrow field might well be ranked highly by his or her immediate management, but no one further up would likely know of the work. Interdisciplinary work, on the other hand, stood out at higher levels because more managers would have seen it. The broader the collaborations, the more managers would know about it. The end result was an organization that strongly favored collaboration and interdisciplinary research. And because the managers who made the decisions had come up through the same process, they were inclined in the same direction.
  • As Doug McIlroy said, “Collegiality was the genius of the system. Nobody’s advancement depended on the relationship with just one boss.”
  • Ken Thompson wrote a significant part of the 1st Unix version in three weeks.
  • This is a good example of a general rule: if a program writes your code for you, the code will be more correct and reliable than if you write it yourself by hand. If the generator is improved, for example to produce better code, everyone benefits; by contrast, improvements to one hand-written program do no improve others. Tools like Yacc and Lex are excellent examples of this rule, and Unix provides many others as well. It’s always worth trying to write programs that write programs. As Doug McIlroy ways, “Anything you have to do repeatedly may be ripe for automation.”
  • How did this relatively small group of researchers from industry manage to produce so many influential books? I can see several reasons. First and foremost, people took writing seriously, they took pains with their own writing, and they were great critical readers of what other people wrote. Doug McIlroy was first among this group; no one else matched Doug’s ability to spot errors …
  • It may sound trivial, but I’m sure that the ability to make revisions so easily led to better writing, because it pretty much eliminated the overhead of making an up to date copy, and it completely eliminated middlemen like typists, editors and printers. Accuracy mattered for technical papers and for the Unix Programmer’s Manual, but control of the whole process especially mattered for books. For programming books, it was vital that the programs were typeset directly from the source code, so we could be sure that what was printed was correct, that it hadn’t been inadvertently changed by human intervention. (Sounds like modern continuous deployment …)
  • It’s worth noting that Sed, Awk, Make, Yacc and Lex all implement some flavor of the pattern-action paradigm. Programs in these languages consist of a sequence of patterns and actions; the basic operation is to check the input against each pattern and when a pattern is matched, perform the corresponding action. … The pattern-action paradigm is a natural way to think about computations that are primarily a sequence of tests and actions.
  • One comment in the Unix source: /* You are not expected to understand this */
  • part of the genius of Ken Thompson and Dennis Ritchie was their tasteful selection of existing good ideas, and their ability to see a general concept or a unifying theme that simplified software systems. People sometimes talk of software productivity in terms of the number of lines of code written; in the Unix world, productivity was often measured by the number of special cases or lines of code that were removed.
  • Pipes do not solve all connection problems, or course, but the fully general non-linear connections of Doug McIlroy’s original concept do not show up often in practice; linear pipelines are almost always good enough.
  • The Unix philosophy certainly doesn’t solve all the problems of programming, but it does provide a useful guide for approaching system design and implementation.
  • As Dick Hamming said, if you don’t work on important problems, it’s unlikely that you’ll do important work.
  • Technical management. Managers have to understand the work they manage. Management in Bell Labs research was technical all the way to the top, so they had a solid understanding of the work both within their own organization and across others. Department heads were expected to know what their people were doing in real detail, not for the purpose of arguing how great it was, but for being able to explain it to others and helping to make connections. At least in 1127 there were no turf wars; it was a cooperative and non-competitive environment where management supported their people, often collaborated, and never competed. I’m not sure that this was a universal experience, but it’s worth aiming for, and doing it well should be part of the reward mechanism for managers.
  • Although Bell Labs management was technically knowledgeable at all levels, AT&T’s upper management seemed removed from new technology and slow to adapt to change. For instance, in the early 1990s, Sandy Fraser, at that time the director of 1127, told AT&T executives that networking improvements would mean that long distance prices would come down from their then-current ten cents per minutes to one cent a minutes, and he was laughed at. Today’s price is pretty close to zero cents per minutes; Sandy was too conservative.
  • Bell Labs management was technically competent, and especially so in 1127, so it could access good work, and it was hands-off, so it didn’t push particular projects or approaches. In over 30 years at the Labs I was never told what to work on. Bruce Hannay, vice president of Research after Bill Baker, said in 1981 in Engineering and Science in the Bell System, “Freedom of choice is of utmost importance to the research scientist, because research is an exploration of the unknown and there are not road maps to tell what course to take. …”
  • The big secret to doing good research is to hire good people, make sure there are interesting things for them to work on, take a long view, and then get out of the way. It certainly wasn’t perfect, but Bell Labs research generally did this well.
  • “What we wanted to preserve was not just a good environment in which to do programming, but a system around which a fellowship could form. We know from experience that the essence of communal computer […] is not just to type programs into a terminal instead of a keypunch, but to encourage close communication.” – Denis Richie, “The Evolution of the Unix Time-sharing System.”
8

What Ken Thompson has been up to at age 80 …

If you are interested in Audio, this may be of interest …