Monday, February 6, 2012

Test is dead vs. ISTQB kills people

Some time ago I wrote this http://www.satisfice.com/blog/archives/664#comment-265390 comment on James Bach’s blog. I would like to open up a bit more the whole theme and how I think about it.
There was a whole lot of discussion of “test is dead” this year. My claim is that “ISTQB kills people”. I don’t even need to use that as a cheap marketing trick adopted from Nietzsche. I don’t need to wear a black robe to say it. The only thing needed is to look news and see when poor software testing causes the loss of lives. (I am not claiming all that is directly because of ISTQB, but I am not too far from the claim either.)

As all clever readers understood, the formulation of the claim is made according to the “test is dead” nonsense. I do criticize ISTQB, and sometimes I do it loud, but surely it’s easy to comprehend I could not possibly have enough information to make claims like that with certainty. It’s a bait, a highly simplified declaration of an extremely complex chain of events. Ironically, that is exactly what ISTQB is doing to testing for example when they define “boundary value testing” like they do.

Many testers are against ISTQB partially because “it’s just a way to make money”. I have nothing against them making money. (I don’t know how much money they make, actually, I only know they don’t make it in Finland) I’d like to have a discussion with someone about this in fact because I think either I am missing something or those testers are; or maybe both. Nevertheless, I am against ISTQB for various other reasons, such as it doesn’t improve skills to think/question (actually quite the opposite), it isn’t about testing skills, it’s based on multiple choice answers and it advocates poor metrics.
I asked also:
How often have you seen a professional use scripts to be able to do his work? They might have some sort of checklists for helping not to forget important things, for example. But if you can imagine some profession where serious pre-scripting is needed, please let me know, so I can think about this again.

And got a reply from Jesper L Ottosen:
Moviemaking. … That is Blockbuster Movie Making.
But then there are also http://en.wikipedia.org/wiki/Dogme_95 movies etc. ;-)
I think this is a good answer for two reasons. Firstly, it’s not accurate in the sense of “serious pre-scripting” (which was not defined by me at all). Secondly, simplification of (blockbuster) movie making can be seen like this, just like is often happening to testing in the eyes of non-testers. When we think about a movie script, we are seeing something that could be thought to be for example user stories. Screenplay can contain for example expressions and movements of actors, but most movies have lightning, sound, cameras, make up, clothing, etc. which are not pre-scripted. But yes, there is indeed a wide variety of movies with a diverse amount of scripting done. I’ve seen quite a few Finnish movies where even the dialogues are not scripted – only some guidelines are given to actors.

Furthermore, I found out this comment by Michael Pilaeten (http://www.pilaeten.be/?p=1622) with Google:
In the comments of one of James Bach’s posts, Jari Laakso claimed that ISTQB is killing people. I can only assume that he’s joking, since no human with an IQ of less than 2 digits would read James’ blog.  But that does not mean that the whole certification discussing has little to no value.

It’s not quite a joke what I wrote, but as I said above, it’s neither a “this is how it is” statement. The essence is showing the foolishness of sentences like “test is dead”. I won’t dig in deeper in this if I don’t get any further comments. However, I would like to take a pick on his analogy between ISTQB certification and car driving license.

I understand how people confuse these to have something in common. At some point I thought this too. I will explain how I don’t see them anymore similar in any way. Please note, I am considering a driving license only from my own context as I have no idea of other driving licenses than a Finnish one. (Ok, I do know how it goes in Romania also, but I haven’t gone through the process personally and I don’t want to use second-hand information.)

ISTQB Foundation Level certification is basically about theory and vocabulary. (Not to mention how far away from reality they go with that.) A car driving license examination is about knowing theory (physics, traffic signs and law) and putting it into practice (driving in traffic, slippery road driving, avoiding obstacles, parking between two obstacles and so forth).

From driving license examination process you get feedback on how well you did, where you can improve, how to lower gas consumption, etc. If you pass the theory part (mandatory before you can even try practical part) you are allowed to try the driving exam. Obviously, you can’t pass the exam if you don’t know how to drive well enough to survive in traffic. (More context, I never did a driving exam is some 1000 person town, but possibly there it’s easier to pass.)

I don’t want to say all certificates and certifications are useless. I don’t even think everything about ISTQB is waste. But as an entity, with all parts included, it is pretty much futile. I hope this clarifies on my current view on the subject. I also hope to get lots of feedback and comments so I can review my own thoughts and get a grasp on how other people see this.

18 comments:

  1. Some hates scripts, some cannot do without them - in context. See http://blog.asym.dk/2011/12/17/about-working-memory-and-testing/
    and
    http://jlottosen.wordpress.com/2012/01/18/the-right-brain-for-the-future/

    ReplyDelete
  2. Thanks for the comment and the links!

    I liked a lot what you wrote earlier on Twitter regarding movies and scripting. Would you like to post a blog entry of that?

    ReplyDelete
  3. Jari, I agree with you that when you say "futile". As you said I too do not say all certifications are useless or I am against certificates. The problem here is, the way they are done is not proper. It has just become a business with no value addition is what I would say.

    Totally, a nice blog post and I hope to see more people blogging about these kind of things. Once I submit this comment, I would share it on how much ever social media websites I can.

    I hope people get the ability to understand the value in something and then go for it rather than just follow something just because it is massively spread with no value.

    -- Santhosh Tuppad
    http://tuppad.com/blog/

    ReplyDelete
  4. Hi Santhosh,

    Thanks a lot for your comment!

    The more noise we make about these certifications, the better it is for the industry. Besides, they (for ex. ISTQB) might even learn something and update their material. :-)

    Thanks again!


    Best regards,
    Jari

    ReplyDelete
  5. I really like your analogy of a tester certification to a driver's license test. In the US, while it varies from state to state, in most cases you have to pass a written test to get a Learner's Permit. That means you know the basic rules and terminology and are allowed to drive under supervision. The full license requires hours of classroom and driving instruction, and passing an actual driving test. I can see the point of having a standardized "tester certification" that is the equivalent of a Learner's Permit; it shows you have know some of the basic concepts and terminology. It might be useful in lieu of previous experience. Anything beyond that is just professional training - nice to have, but no substitute for experience, and no proof of ability.

    ReplyDelete
  6. Hello Sean,

    I appreciate your comment and insight to the analogy!

    The Learner's Permit you mention seems like a really good idea. I wish something similar would be applied more widely.


    Best regards,
    Jari

    ReplyDelete
  7. Jari,

    Nice post.

    To me ISTQB is an epidemic and I guess the only way we could fight against it is when more testers condemn it publicly. I have come across many testers who do this certification because their career managers, company, recruiter asked them. It's strange how none want to do it for their own but are forced to do something.

    It's a shame and I hope this changes soon.

    Regards,
    Sharath
    http://testtotester.blogspot.com/

    ReplyDelete
    Replies
    1. What I like about your comment is that it's an epidemic.

      In the nowadays fast growing software pushed society it's a must that software gets tested thoroughly. In order to be able to do so people need to know the basic terminolgy. Especially in project environments where most of the (project)people don't understand the importance, positives thus the need of testing, etc.

      The way I look at it:
      Everybody is responsible for his own.
      In order to better/enhance your craft you should get the knowledge and experience where/whenever you can.
      There will always be companies who will benefit from it.

      We have the responsibilty to make sure that we get what we need!!

      Best regards,
      Chester

      Delete
  8. Hi Sharath,

    Good to hear there are more who think alike. Thanks!

    I've added your blog on my read/follow list.

    Have a good day!


    Best regards,
    Jari

    ReplyDelete
  9. Well, I'm a test lead in charge of 18 testers. 3 of them have already taken ISTQB foundation level, and about 3 more want to take it. That's a third of my testers.

    Being quite pragmatic, I'm against all sort of certification (although I don't forbid them, since I'm getting certified as a Bachelor's in Computer Science myself, go figure). In every Personal Development Plan I encourage people getting certified to not just study for the questions, but to acquire certain skill. At least some.

    Regarding test being dead, I wrote a post about it in my blog. Basically, the kind of answer I got to it was, well, Google may get away with it because it has a large user base, but they still have to do usability, performance and other types of testing. Also, there are other types of apps that need to remain confidential while being developed. There'll be room for testing in those also, for a long time!

    ReplyDelete
    Replies
    1. Thanks for the comment, Omar!

      I am only against bad certifications. There are some really good ones too.

      I don't know how you see the "test is dead" claim, but what I noticed (online) was most people didn't get it at all. It's like The Death card in Tarot; it doesn't mean dying/disappearing, but a transition.

      There are many contexts for testing and definitely "working in Google" is not something that can be generalized to *all* software testing. James Whittaker likes to make generalizations and troll a bit on the side. This is his trademark.

      I agree completely with you. There will be a lot of testing in the future. Actually, I think the amount of testing needed is growing heavily in the future. We are moving to a world more and more controlled by devices and software.


      Best regards,
      Jari

      Delete
  10. Blog post done on movies here:
    http://jlottosen.wordpress.com/2012/04/17/small-changes-inbig-scripts/

    ReplyDelete
    Replies
    1. Hi Jesper,

      Thanks for writing the post and leaving the link here! I've seen some inspiring movies where they didn't have a manuscript. One I saw during my last trip to Finland and the actors said there was only a set of ideas the director wanted actors to think in each scene. I think that is an interesting idea and it was great fun to see the movie.

      Have a great spring!


      Best regards,
      Jari

      Delete
  11. Hey, nice site you have here! Keep up the excellent work!

    ISTQB

    ReplyDelete
  12. Hi,

    I think, that the ISTQB-Standard is good and helps people, who are new to testing, learning a structured approach. Most people in my company didn't had this background, so with some techniques from ISTQB, I could help them, to write better an more structured test cases for example.

    I also see, that people want to make money with ISTQB. As a founder of a !totally free! German learning plattform under http://smartwebapps.de, I made the sad experience, that people don't like it, if the learners get free access to a training playground and propably don't join their expensive trainings.

    Because it was annoying for me, I help spreeding the word, that you can make it with a good book and appropriate trainings questions through the test. When you have that view on it, then nothing is bad at ISTQB, because people really learn the topics step by step, which helps them keeping things in mind.

    http://smartwebapps.de/istqb-im-selbststudium-bestehen-geht-das

    Best regards,

    Stefan

    ReplyDelete
    Replies
    1. Hi Stefan,

      Thanks for the comment.

      Can you give examples of better and more structured test cases? Without more details, I can't comment on that.


      Best regards,
      Jari

      Delete
  13. Best book to learn software testing and ISTQB Advanced Certification is Advanced Test Strategy. It is available in Flipkart and Amazon as well!! URL:
    http://www.amazon.com/Advanced-Test-Strategy-Foundation-Questions/dp/1482812223

    ReplyDelete
    Replies
    1. I published this comment as an example of what comments will not be published in the future. At very least, you must have reasons why you market this book (of nonsense).

      Delete