Testing scientifically - Differential diagnosis in testing (troubleshooting scientifically)

Everybody lies
you know perfectly well where this comes from


Sometimes your testing goes smoothly and dully, but sometimes you just can't make head or tail of what you observe. It feels like there is some system behind but it is quite complex as if more that one factor were influencing the result. Sometimes it is extremely useful to realize there may be more than one problem behind. Luckily, testing was not created with the software industry so we may go for help to some older sciences.

In the medical world there is a tool known as differencial diagnosis or DDx[5]. It employs four basic steps that help to take our big chaos and sort it into several meaningful lumps. Having been translated from wikimedical to software testing language these basic steps[6] will look like this:

1. Find out all info about your piece of software (i.e. requirements, configuration etc)
2. Make up a list of observed symptoms (not just where it contradicts requirement but full description of actual behavior)
3. Make up a list of possible causes (aka "candidate conditions")
4. Prioterize these causes by serverity (i.e. impact or how it affects interested parties)
5. Eliminate causes one by one until you get the botton of it

Note: remember that there may be more than one cause. The fact that some cause was found but it did not explain all the symptoms may happen because
-- there is another cause responsible for the rest of symptoms
-- your guess is wrong

Note: sometimes you may get no result, it might mean that something was not taken into consideration or we do not posess all the required information

Note: Occam's razor may be of some help as well: "When you hear hoofbeats, look for horses, not zebras" [7]


--------------

Footnotes:
[5] Differencial diagnosis at wiki: <a href="en.wikipedia.org/wiki/Differential_diagnosis">link to the article</a>

[6] Original description of steps from wiki (same as [5]):
Differential diagnosis has four steps.
First, the physician should gather all information about the patient and create a symptoms list.
Second, the physician should make a list of all possible causes (also termed "candidate conditions") of the symptoms.
Third, the physician should prioritize the list by placing the most urgently dangerous possible cause of the symptoms at the top of the list.
Fourth, the physician should rule out or treat the possible causes beginning with the most urgently dangerous condition and working his or her way down the list.

"Rule out" practically means to use tests and other scientific methods to render a condition of clinically negligible probability of being the cause. In some cases, there will remain no diagnosis; this suggests the physician has made an error, or that the true diagnosis is unknown to medicine. Removing diagnoses from the list is done by making observations and using tests that should have different results, depending on which diagnosis is correct.

[7] Original quote from wiki (same as [5]):
As a reminder, medical students are taught the adage, "When you hear hoofbeats, look for horses, not zebras," which means look for the simplest, most common explanation first. Only after the simplest diagnosis has been ruled out should the clinician consider more complex or exotic diagnoses.