|<<>>|36 of 214 Show listMobile Mode

No Silver Bullet—Essence and Accident in Software Engineering by Frederick P. Brooks, Jr. (1986) (read in 2019)

Published by marco on

Disclaimer: these are notes I took while reading this book. They include citations I found interesting or enlightening or particularly well-written. In some cases, I’ve pointed out which of these applies to which citation; in others, I have not. Any benefit you gain from reading these notes is purely incidental to the purpose they serve of reminding me what I once read. Please see Wikipedia for a summary if I’ve failed to provide one sufficient for your purposes. If my notes serve to trigger an interest in this book, then I’m happy for you.

This is a relatively short but important essay in the world of software engineering theory. It’s not really for programmers who’ve stumbled over from scripting Photoshop or who’ve decided that there’s good money in copy/pasting code that they don’t understand from StackOverflow. The audience is more self-selecting. If you’re likely to read an essay with this name, then you’ll likely be receptive to its ideas. That is, if you don’t already agree with the premises in the document, you’re unlikely to be convinced by it.

Brooks is one of the inventors of OS/360 for IBM mainframes and, more famously, the author of the book The Mythical Man-Month. He draws on a lot of experience when he writes that the difficult bit of software is not writing it so that it works—it’s figuring out what you want to write in the first place.

“I believe the hard part of building software to be the specification, design, and testing of this conceptual construct, not the labor of representing it and testing the fidelity of the representation.”

He argues in this paper that where we often go wrong is when we attempt to “abstract away […] complexity” but end up “abstract[ing] away its essence”. Abstractions are a good thing. The only way to represent any process in software is to create a model of it. But we haven’t historically been careful enough that what we ignore—the corners we cut, the values we round—aren’t part of the essence of what we’re building.

This leads Brooks to the next conclusion, that, “No facilitation of expression can give more than marginal gains.” If you want to write software more quickly, changing languages or runtimes or patterns will offer, at best, marginal gains relative to improving the process you use to design that software and gather and define its parameters and requirements.

This is a tragic conclusion for most programmers, a vast majority of whom are much more interested in trying out new techniques and languages and IDE tools and extensions and just tweaking the hell out of the implementation side of things. This is all very interesting and can lead to fruitful gains. Brooks acknowledges as much,

“The gap between the best software engineering practice and the average practice is very wide—perhaps wider than in any other engineering discipline. A tool that disseminates good practice would be important.”

But once you’re testing properly and have a good framework and a good editor and you’ve got CI and maybe even CD, there’s no more room for quantum leaps in improvement. At that point, you can only get significantly better and faster by writing only the code that you actually need. For that, you need to optimize defining your requirements.

Grasping and defining requirements is by no means an easy thing, as any non-trivial software involves a state machine with exponentially increasing combinations of states.

“From the complexity comes the difficulty of enumerating, much less understanding, all the possible states of the program, and from that comes the unreliability.[1]

Where does this complexity come from? Often, from outside of the system, at the edges, where the software must interface with other systems. Perhaps these are legacy systems; they are almost certainly less flexible than the software being designed and written right now. The new software is, by definition, more malleable than software or processes already in production. As Brooks puts it,

“[…] all cases, much complexity comes from conformation to other interfaces; this cannot be simplified out by any redesign of the software alone.”

All of this necessary/essential complexity “makes personnel turnover a disaster” once you’ve trained someone to understand it. This is a lesson that the software world—with its focus on exchangeable resources that just provide hours of work—has never learned or, more generously, forgotten.

Perhaps this is due to the prevalence of so many layers of management. They generally don’t know how to do anything special and are generally highly interchangeable with other managers who also aren’t very special. Their ego depends on their worldview considering all other people to be the same. This includes the highly trained and skilled staff who they manage.

Brooks ends with an eloquently and succinctly stated summary that jibes 100% with my experience over the last 25 years of designing and building software.

“Therefore the most important function that software builders do for their clients is the iterative extraction and refinement of the product requirements. For the truth is, the clients do not know what they want. They usually do not know what questions must be answered, and they almost never have thought of the problem in the detail that must be specified. (Emphasis added.)”

This is what we have to work with: the domain specialist (the client) doesn’t have the know-how to even know how to describe the domain. Sometimes there is no real domain specialist; there is just someone with money and a vague idea or, even worse, someone who thinks that they are a domain expert. The job of a software designer is to become (enough of) a domain specialist to bridge the gap. If nobody bridges the gap, then the software will fail.


[1]

This reminds me a bit of an article I read called Text Editing Hates You Too, which included the following:

“The necessary complexity here is immense, and this post only scratches the very surface of it. If anything, it’s a miracle of the simplicity of modern programming that we’re able to just slap down a