No theory forbids me to say "Ah!" or "Ugh!", but it forbids me the bogus theorization of my "Ah!" and "Ugh!" - the value judgments. - Theodor Julius Geiger (1960)

Rational Accidents

John Downer has written this great book about technology, reliability, and risk in modern society. He has done a great job in clearly describing the Fukushima disaster and aviation accidents Aloha Airlines Flight 243 and the 737-MAX crashes. Importantly, he shows that the unprecedented reliability of nuclear reactors and jetliners is always accompanied by inherent uncertainty.

Through reading the book, I learned how the aviation industry achieves extraordinary safety levels through real-world data, recursive learning, and design stability. Although there are huge economic incentives behind civil aviation’s commitment to safety, Boeing allowed profit motives to compromise safety, see the 737-MAX crashes. Boeing’s decision not to add redundancy to certain systems, driven by cost concerns, resulted in fatal consequences, proving that safety must always remain the top priority.
The aviation industry’s backup systems have helped in achieving a high level of safety. But, even with redundant systems, unexpected failures can still occur: e.g. simultaneous engine loss or cascading system failures.

So, technological accidents aren't simply all due to human error at the sharp end. Interactions that were not part of the engineering assumptions can lead to accidents. For instance, a section of the fuselage of Aloha Airlines Flight 243 tore away mid-flight due to stress fractures and metal fatigue.
Fukushima Daiichi's nuclear plant failed catastrophically when the actual tsunami far exceeded design assumptions.

Downer calls nuclear reactors and jetliners catastrophic technologies: these are required to perform at extraordinary levels of reliability. Despite expert assurance that failure is nearly impossible, we see from the example of Fukushima disaster that the possibility of catastrophic failure is always present, no matter how improbable.

Downer writes that technological mastery has its limits, and overconfidence in our ability to predict and control outcomes can lead to catastrophic consequences. The book is a reflection about the complexity of reliability, and the fragility and resilience of systems we depend on.

 

Downer, J. (2024), Rational Accidents: Reckoning with Catastrophic Technologies, MIT Press.