Wednesday, October 4, 2017

Test Failure and Test Blocker



Test Blocker

Blocking QA from testing may just require a small code fix or may just need a fix to the QA env configuration.

It affects the QA testing and affects the QA testing schedule and productivity.  Unblocking QA is critical and needs to be treated as an urgent issue.



Unblocking QA may not need to wait for a build.  Build schedule is determined by readiness of dev, which may still be working on the code changes or performing unit testing.  Pushing dev to provide a build for "unblocking QA" is actually dangerous as you are pushing more incomplete work or more work that are done in a rush.  It actually may introduces more blocking issues.

Unblocking QA can be done in a hotfix but the hotfix needs to go away in the next build.  Sometimes, after an issue got unblocked and it becomes lower priority.  It is still critical to follow up as you do not want to keep applying a workaround.

Getting a fresh QA env for every build is ideal but it requires the QA testing technology that can bring the testing data back.

Refreshing the QA env periodically not for every build may be more realistic.

You need to be careful when you apply a workaround (or a hotfix) to unblock QA.  Sometime, the workaround may make the testing meaningless.


Test Failure 

Test failure means that the result of testing does not meet the expected result.

The expected result can be simply
  • I can login
  • I can see the next page when I click on a button or a link.
  • I can save my change
  • I cannot save my change as I have not fill the blank or I did not enter data correctly.
  • I should not be allowed to enter a page if I am not granted with required permissions

Sometime testing failure and testing blocker is the same.  You failed in the early workflow such "I cannot login" and you are blocked as you cannot complete the workflow.

However, blocked in a workflow does not mean that you cannot continue working.  If a QA resource can work on something else, the QA may not actually be blocked.

Whether QA is blocked can be considered from two angles:

  • QA resource is wasted as they cannot do anything
  • QA resource is wasted on working on some less critical activities
***






When you require people to work overtime, you may find that your team has more chances to be blocked.  These are the  consequences:

More people need to work overtime to unblock
They go back to take rest and come back in regular hours
In any case, relying on people to work overtime or working over weekend is problematic.  It cannot be a long-term strategy.  When you keep mentioning "war room". Guess what, every regular room becomes a "War Room".

When every bug is P1, there is no P1.  You can keep introducing P0, blocker, etc.  Whatever.  Asking employees to cover your own failure in planning and setting unrealistic target is not the way how software development should work.










No comments: