ISTQB is a bad bad thing

ISTQB is a bad, bad thing. Reeeeeally bad thing. And here is why.

First of all, they certify something there are more than one opinion about. Something that has different approaches. Something that is poorly measurable since it depends on efficiency of decision making process. And decades of IQ testing and similar attempts to measure and control human brain and psyche proved it spectacularly inefficient.

Whatever you can read in ISTQB material is just an opinion and a book would suffice to share it. Unfortunately, royalties that only one or even several books can bring is nothing compared with certification system would bring. Basically, you bypass market competition by forcing people into buying books and paying for exams, and you do it by pressuring their employers into pressuring their employees.

IT suffers from its own popularity

IT suffers from its own popularity. People go to IT not only because they love it or know how to do it, but because they need money, a reputation and a little bit of power. Because of this, a significant part of decisions are made out of greed, vanity and for the sake of a line in a resume. And an attempt to change priorities in favor of the interests of the project often leads to a conflict. The problem is equally common among both management and engineers, although it manifests itself in each group in its own way

2022-09-09

Flow for Managers

There's no such a thing as a universal way to do your job and not let it kill you.

To achieve the best results some professions need to work in a flow, and some -- in a quick succession of tense-release phases. And, among other things, 'the result' is assessed by how much better your life has become (or maybe it took a turn for the worse).

It looks like Pomodoro-like techniques are best for managers who are forced to operate in a quickly changing environment, while flow is necessary for engineers and researchers.

Are you sure it's your job that you hate

It is important and popular fact that things are not always what they seem
Hitchhicker's Guide to the Galaxy, the movie

Years of employment kind of proved that one's attitude to their job is not necessarily connected with an objective quality of that job. After working for many years as an employee myself, and after talking to the other employees, I was surprised to discover that the job is almost never an evil in itself, but is often treated as an underdog that can't say anything in it's defense. 

Probably it is so because its advocates are managers, and who trusts managers? Right. Even managers don't trust managers. (Seriousely though, managers are not bad because they are managers, this problem of trust comes from the difference of understanding and a long history of miscommunication that exists between different social groups).

This is not an article on phychology (and I'm not a professional psychologist), but being familiar with such notions as psychological defences and rationalization as a special case of it may be of help. To say it plainly, these defences came to life with the best of intentions, and as many other originally good things got a bit rogue in a course of time. And, in attempt to keep you safe, integral and unfamiliar with pain, they may slightly distort your view of yourself, your priorities and your life. Thus preventing you from being able to get to the root cause and fix it, and hurting you on the way.

So here are some of my favorite job-related biases and the reasons behind them as I see it.

Troubleshooting checklist

Anybody, who is facing a necessity to troubleshoot, hopes for a miracle. 

For something like closing your eyes and seeing the problem just solving itself without much fuss or explanation, when you open them. We all know this feeling. We all know it almost never works this way, because there is no such a thing as a miracle. 

But there are causes and effects instead. And some brain energy is required to make sense of what you see and to get out of the entanglement. And that brain energy is something that is taken for granted but is not that available.

Those little gray cells

Brainwork is something we think is always there and always will be. You probably no longer believe that storks bring children[1], but for some reason you are sure that your brain is a kind of perpetuum mobile which functions under any condition and knows nothing about sabotage.

That's a lie. It does. And a good (really boring but really good) book[2] by Daniel Kahnneman reminds you a good number of times that the rational, non-automatic, weighing part of your mind is lazy (which is where Kahneman ends and general phycology begins) and being lazy means that your resource is far from enough.

What if test certification systems put us in a box to think inside it?

With vertical thinking one uses the negative in order to block of certain pathways. With lateral thinking there is no negative.
Edward de Bono, Lateral Thinking

Sometimes I get those anarchistic ideas that may, if voiced at inappropriate moment, seriously damage your reputation, unless you put them in a nice colored gift package of justification and proof. And though some may see this as creating a Pandora box, at the bottom there is always hope of finding an important clue or even leading yourself out of the dead end.

Today's one is the following. Let's say we limit access to any exams or certification systems for any tester with relevant hands-on experience under 2 or 3 years? The same way they do for MBA or certain levels of trainings for the system administrators? Mind you, not information, or books, or trainings, but certification. Because I strongly believe there is a huge difference between critically reading a book and drilling something in order to check certain boxes in test. 

Even if you are fully conscious about the fact that your own opinion is different. Or (which is worse) if you have no opinion and take things on faith. Testing is an activity of the scientific type, not a religious practice, and we need to be extremely careful with faith and our ability to say what you think. Test practice proves that a good tester needs to be honest and courageous, because sometime this is exactly what makes the difference between true and false.

Testing scientifically - Differential diagnosis in testing (troubleshooting scientifically)

Everybody lies
you know perfectly well where this comes from


Sometimes your testing goes smoothly and dully, but sometimes you just can't make head or tail of what you observe. It feels like there is some system behind but it is quite complex as if more that one factor were influencing the result. Sometimes it is extremely useful to realize there may be more than one problem behind. Luckily, testing was not created with the software industry so we may go for help to some older sciences.

In the medical world there is a tool known as differencial diagnosis or DDx[5]. It employs four basic steps that help to take our big chaos and sort it into several meaningful lumps. Having been translated from wikimedical to software testing language these basic steps[6] will look like this:

1. Find out all info about your piece of software (i.e. requirements, configuration etc)
2. Make up a list of observed symptoms (not just where it contradicts requirement but full description of actual behavior)
3. Make up a list of possible causes (aka "candidate conditions")
4. Prioterize these causes by serverity (i.e. impact or how it affects interested parties)
5. Eliminate causes one by one until you get the botton of it

Note: remember that there may be more than one cause. The fact that some cause was found but it did not explain all the symptoms may happen because
-- there is another cause responsible for the rest of symptoms
-- your guess is wrong

Note: sometimes you may get no result, it might mean that something was not taken into consideration or we do not posess all the required information

Note: Occam's razor may be of some help as well: "When you hear hoofbeats, look for horses, not zebras" [7]


--------------

Footnotes:
[5] Differencial diagnosis at wiki: <a href="en.wikipedia.org/wiki/Differential_diagnosis">link to the article</a>

[6] Original description of steps from wiki (same as [5]):
Differential diagnosis has four steps.
First, the physician should gather all information about the patient and create a symptoms list.
Second, the physician should make a list of all possible causes (also termed "candidate conditions") of the symptoms.
Third, the physician should prioritize the list by placing the most urgently dangerous possible cause of the symptoms at the top of the list.
Fourth, the physician should rule out or treat the possible causes beginning with the most urgently dangerous condition and working his or her way down the list.

"Rule out" practically means to use tests and other scientific methods to render a condition of clinically negligible probability of being the cause. In some cases, there will remain no diagnosis; this suggests the physician has made an error, or that the true diagnosis is unknown to medicine. Removing diagnoses from the list is done by making observations and using tests that should have different results, depending on which diagnosis is correct.

[7] Original quote from wiki (same as [5]):
As a reminder, medical students are taught the adage, "When you hear hoofbeats, look for horses, not zebras," which means look for the simplest, most common explanation first. Only after the simplest diagnosis has been ruled out should the clinician consider more complex or exotic diagnoses.

Testing scientifically - Scientific method in testing

Scientific method in testing

begins here

Aristotle is believed to think that women had less teeth than men. At the moment we can't say for sure if he could not count or women in Greece of his time experienced problems with nutrition.
A fact.

To tell the truth best ever definition of scientific method is already provided by wikipedia and according to it

-- method should be empirical and have measurable evidence [2]
-- method should be proving or disproving a theory/assumption [3]
-- approach and analysis should be as unbiased as possible [4]
-- knowledge should be documented and sharable [4]
-- results should be reproducible [4]

All these map on what we know about testing perfectly. In fact any good testing professional has been doing this for years but did not always realise that there was a solid scientific background all the time. Let's look at each of the above points closely.

Well, being empirical. Unless you are in SQA department your job 100% emprirical anyway.

Measurability. In this case measuring is not about centimeters, but should be used in a bit wider sense, say, you do not expect your test to return some ok result, you expect it to return either specific number of lines, or no error message, or what not. But you need to be specific.

Being an applied technique testing is quite remote from most of theories and deals with more low-level thing like requirements and assumptions (derived directly from said requirements). As such your test is supposed not just to do something abstract but to either prove or disprove an assumption. Suppose that under certain circumstances some button is  expected to tunr blue. Assumptions are:
-- button must look blue given preconditions are met
-- button should not look non-blue
-- etc

It is possible to talk endlessly on human biases how testers being human suffer from them severely. Here are some of them
-- fear of looking incompetent (and not reporting a potential issue)
-- fear of aggressive reactions (and not reporting a potential issue)
-- being lazy to deal with consequences (and singing it off in hope that by the moment it fires you'll be far away)
-- etc

Sharing knowledge. Ahh. Knowledge means power and unless you are lucky enough to work for a company with right corporate culture you will have to beg for it until you build your own informational capital and be able to trade. There is a lot of literature on business and functional analysis with a log of explanations of why documenting is important so I'll do it briefly here:
-- if information is documented and shared you do not have to waste anybody's time and effort to get it
-- if information is documented and shared it is less likely to be lost if key persons leave
-- if information is documented and shared it is much easier to make sure your tests are well aligned with requirements
-- etc

Reproducible results. Tiny but crucial difference between a bug and a glitch. To prove there is a bug it is important to be able to show that a very specific cause leads to a very specific result. Being inaccurate in this usually leads to non-reproducibles and I-do-not-wanna-know-it-works-on-my-machine stuff. Golden rule: make sure to reproduce it twice at least before you claim it works ok or it is a bug. As I mentioned earlier there are both false positive and false negative pitfalls.

-------------

Footnotes:
[2] "To be termed scientific, a method of inquiry must be based on empirical and measurable evidence subject to specific principles of reasoning." Source: http://en.wikipedia.org/wiki/Scientific_method

[3] "The chief characteristic which distinguishes the scientific method from other methods of acquiring knowledge is that scientists seek to let reality speak for itself, supporting a theory when a theory's predictions are confirmed and challenging a theory when its predictions prove false." Source: http://en.wikipedia.org/wiki/Scientific_method

[4] "Scientific inquiry is generally intended to be as objective as possible in order to reduce biased interpretations of results. Another basic expectation is to document, archive and share all data and methodology so they are available for careful scrutiny by other scientists, giving them the opportunity to verify results by attempting to reproduce them." Source: http://en.wikipedia.org/wiki/Scientific_method

-------------

To be continued in part 3

Testing scientifically - Testing and quality

Testing and quality

Quality is conformance to requirements, not 'goodness' or 'elegance'
usually attributed to Phil Crosby


Testing is measure taken in order to make sure you get exactly what you expect. Or very much like it. Or has some of expected features. At least. Testing deals with how you put your expectations into words and as such is closely connected with message-sent-is-not-the-same-as-message-received thing. This is where requirements come in and analyst plays his/her part.

Ok. Suppose we agreed on meaning of words and shared goals, how do we prove we got what we wanted? And this is where testing starts and test specialist appears on the scene. In an everyday common context test stands for "checking". In a narrower scientific context it implies a number of actions that prove some statement. Do you remember functional logic? If something in a statement contradicts actual reality or common sense then statement is considered to be false. Otherwise it is said to be true. It works pretty similarly with the testing except that reality and quite often common sense are replaced with requirements. If test results contradict requirements then test is failed. If test result are in perfect harmony with those requirements then test is passed or, in other words, we got what we wanted (or agreed to think that we wanted this).

NB: Sometimes results turn out ot be false positive or false negative. This may happen due to a number of reasons like poor understanding of what is going on, ambiguos requirement or something being wrong with the test design. We are dicussing this a bit later.

Roughly all tests may be split into two categories: tests that try to confirm that anything works alright and tests that try to prove there is something wrong. Scientifically these two approaches are called verification and falsification and their history goes as far back as Karl Popper's writings and even further then that. It is important to remember that this in no way contradicts or replaces specifict methods of building test coverage such as equivalence partitioning, boundary values analysis, cause and effect, error guessing or exhastive testing.[1]

Why mess with science when the only thing we want is to make sure it works during sales demo? In fact this question is crucial and the answer is simple. If your goal is just to sell something that somewhat works then testing is not necessary. Testing is only required if you expect to get something specific and getting what you expect constitues quality. In case you need quality messing with science is unavoidable.

-------------

Footnotes:
[1] Based on this page (Russian with automatic traslator): About testing

-------------

To be continued in part 2