I was asked to write up such a document for an incident where our team had written a new feature which, upon launch, did absolutely nothing. Our team had accidentally mistyped a flag name on the last day before we handed it to a test team, the test team examined the (nonfunctional) tool for a few weeks and blessed it, and then upon turning it on, it failed to do anything. My five whys document was most about "what part of our process led to a multiweek test effort that would greenlight a tool that does nothing that it is required to do."
I recall my manager handing the doc back to me and saying that I needed to completely redo it because it was unacceptable for us to blame another team for our team's bug, which is how I learned that you can make a five why process blame any team you find convenient by choosing the question. I quit not too long after that.
A few problems I faced:
- culturally a lack of deeper understanding or care around “safety” topics. Forces that be inherently are motivated by launching features and increasing sales, so more often than not you could write an awesome incident retro doc and just get people who are laser focused on the bare minimum of action items.
- security folks co-opting the safety things, because removing access to things can be misconstrued to mean making things safer. While somewhat true, it also makes doing jobs more difficult if not replaced with adequate tooling. What this meant was taking away access and replacing everything with “break glass” mechanisms. If your team is breaking glass every day and ticketing security, you’re probably failing to address both security and safety..
- related to the last point, but a lack of introspection as to the means of making changes which led to the incident. For example: user uses ssh to run a command and ran the wrong command -> we should eliminate ssh. Rather than asking why was ssh the best / only way the user could affect change to the system? Could we build an api for this with tooling and safeguards before cutting off ssh?
So I did what they wanted and the root cause was:
On December 11 1963 Mr and Mrs Stanley Smith had sexual intercourse.
I got asked what that had to do with anything and I said, "If you look up a few lines you'll see that the issue was a human error caused by Bob Smith, if he hadn't been born we wouldn't have had this problem and I just went back to the actual conception date."
I got asked how I was able to pin it to that date and said "I asked Bob what his father's birthday was and extrapolated that info"
I was never asked to do a RCA again.
Causal Analysis based on Systems Theory - my notes - https://github.com/joelparkerhenderson/causal-analysis-based...
The full handbook by Nancy G. Leveson at MIT is free here: http://sunnyday.mit.edu/CAST-Handbook.pdf
Someone said the quiet part loud! :
"""
Common circumstances missing from accident reports are:
Pressures to cut costs or work quicker,
Competing requests for colleagues,
Unnecessarily complicated systems,
Broken tools,
Biological needs (e.g. sleep or hunger),
Cumbersome enforced processes,
Fear of being consequences of doing something out of the ordinary, and
Shame of feeling in over one’s head.
"""
Systems do not have to facilitate operators in building accurate mental models. In fact, safe systems disregard mental models, because a mental model is a human thing, and humans are fallible. Remove the human and you have a safer system.
Safety is not a dynamic control problem, it's a quality-state management problem. You need to maintain a state of quality assurance/quality control to ensure safety. When it falters, so does safety. Dynamism is sometimes not a factor (although when it is, it's typically a harmful factor).
Also fwiw, there's often not a root cause, but instead multiple causes (or coincidental states). For getting better at tracking down causes of failures (and preventing them), I recommend learning the Toyota Production System, then reading Out Of The Crisis. That'll kickstart your brain enough to be more effective than 99.99% of people.
> If we analyse accidents more deeply, we can get by analysing fewer accidents and still learn more.
Yeah, that's not how it works. The failure modes of your system might be concentrated in one particularly brittle area, but you really need as much breadth as you can get: the bullets are always fired at the entire plane.
> An accident happens when a system in a hazardous state encounters unfavourable environmental conditions. We cannot control environmental conditions, so we need to prevent hazards.
I mean, I'm an R&D guy, so my experience is biased, but... sometimes the system is just broke and no amount of saying "the system is in a hazardous state" can paper over the fact that you shipped (or, best-case, stress-tested) trash. You absolutely have to run these cases through the failure analysis pipeline, there's no out there, but the analysis flow looks a bit different for things that should-have worked versus things that could-never-have worked. And, yes, it will roll up on management, but... still.
Fantastic RCA: Remove requirement that caused the action that resulted in the problem occurring.
Bad RCA: Lets get 12 non technical people on a call to ask the on call engineer who is tired from 6 hours managing the fault, a bunch of technical questions they don't understand the answers to anyway.
(Worst possible fault practice is to bring in a bunch of stakeholders and force the engineer to be on a call with them while they try and investigate the fault)
Worst RCA: A half paragraph describing the problem in the most general terms to meet a contractual RCA requirement.
If the result/accident is too bad though you need to find all the different faults and mitigate as manyias possible the first time.
People rarely react well if you tell them "Hey this feature ticket you made is poorly conceived and will cause problems, can we just not do it?" It is easier just to implement whatever it is and deal with the fallout later.