Monoliths and Monorepos

Software
, ,
1

Software systems are complex. There are many ways to manage this complexity. One approach might be to break the software into many small pieces stored in many repositories. I once thought this. With inexperience, this is natural – most of the tech chatter we hear today is about micro-services, distributed systems, and the various problems of scaling software (Docker, Kubernetes, etc). However, most of us are not Google, and many of us are solving problems where one or two cloud servers can easily handle the device/user load. In the early days of product development, time is much better spent on solving the user’s problems as simply as possible and doing things that don’t scale. Even creating too many directories in your source code early on can be a mistake – especially with typed languages that are easy to refactor. If the product is successful, there will be time/resources later to figure out how to make it scale.

There are some systems that are inherently distributed. Two examples that come to mind are browser front-ends and IoT systems. The computing happens in physically different locations, so there is no way to avoid being distributed. There may also be systems where you need to blend vastly different technologies written in different languages (such as machine learning). In these cases, distributed technologies such as Protobuf and Nats.io are very useful. There are also cases where you want to make some code public/OSS and some private – in this case, multiple repositories may make sense. Large organizations and massive scale have challenges that lend to splitting things up. However, there are also large projects (such as the Linux kernel) that have done very well as a monolithic project in a monorepo. Does your project have 28 million lines of code?

Working with technologies (such as Go, Rust, and Elm) that don’t require complex build/distribution tooling brings some of the joy back into programming. The best solution to many problems is to make things simpler, not add layers of complexity and tooling.

Below are my notes on these topics:

2
3

The Tanenbaum-Torvalds Debate

Summary:

This is Appendix A of O’Reilly’s Open Sources (1999) — the archived comp.os.minix threads from early 1992 known as the “Linux is obsolete” debate.

The opening salvo. On January 29, 1992, Andrew Tanenbaum posted that Linux was architecturally backwards on two counts. First, microkernels had won the design argument among people who actually build operating systems, and Mach 3.0 benchmarks had removed performance as the last excuse for monolithic kernels; writing a monolithic kernel in 1991 was therefore a step back into the 1970s. Second, Linux was welded to the 80x86, which he expected RISC to displace — MINIX had already been ported to 68k, SPARC, and NS32016. His closing advice was that anyone wanting a modern free OS should look at something microkernel-based, like GNU.

Linus fires back. Torvalds replied the same day, conceding that microkernels are theoretically and aesthetically nicer but arguing that Linux won on being available now — had the GNU kernel been ready the previous spring, he wouldn’t have started. He countered that MINIX doesn’t do the microkernel thing well anyway (single-threaded filesystem, race conditions in the multithreading hacks), and that portability matters at the API level, not in the kernel: exploiting the 386 aggressively is what made a much simpler design possible in the first place. He apologized the next day for the tone, calling it his first and hopefully last flamefest.

The exchange continues. Tanenbaum defended MINIX’s limits as deliberate — it had to run on a diskless 4.77 MHz PC so students could afford it — dismissed multithreaded filesystems as a performance hack, and got in the line about Linus not getting a high grade for the design. Linus replied that a multithreaded kernel falls out of the monolithic design for free rather than being a hack, and noted that the entire Linux kernel source was smaller than Mach’s i386-specific code alone.

Everyone else piles in. Ken Thompson agreed microkernels were probably the future but noted monolithic kernels are easier to implement and easier to turn into a mess. Ted Ts’o pushed back on the microkernel-consensus claim, citing arguments for keeping the filesystem in the kernel and context-switch costs in OSF/1 Mach, and said he wanted a real OS rather than a research toy. Others reported that MINIX’s single-threaded FS made background work unusable in practice; Peter da Silva defended microkernels by pointing to AmigaOS as a shipping, fast one.

The pivot to licensing. Tanenbaum’s “Unhappy campers” post identified three sore points — monolithic vs. micro, portability, and whether software ought to be free — and argued the free-ness complaint was emotional, since MINIX shipped with source and cost about $60 effectively. His deeper point was that he refused feature contributions to keep the system teachable, and that coordination, not copyright, is what limits distributed development: one surgeon, not a hog-butchering team. He asked whether Linus would let Linux escape his control. Linus answered flatly that he wouldn’t try to keep control, that his influence came only from knowing the code best, and that the copyright’s real function was preventing others from selling his work without source — the thing MINIX’s license made impossible, forcing would-be improvers into endless patch-on-patch distribution.

The threads wind down with a widely-shared conclusion: the two systems had different purposes, MINIX as a documented teaching example and Linux as a working Unix for people who wanted to get things done. In hindsight the licensing argument turned out to be the consequential one, and Tanenbaum’s own framing about coordination was disproven fairly thoroughly by what followed.