Thursday, April 23, 2015

Automated Testing - what to be automated?

Automated Testing helps improving QA's productivity.  It automates the jobs that, otherwise, need to be done manually.

What are the jobs to be automated?

What are QA's jobs?

QA identifies the tasks that the users of the software will do.  The QA executes the tasks with the assumptions that the software will be used under certain circumstances .  The QA checks if the tasks can be done and whether the results are expected.  Before QA performs the tasks, the QA identifies the expected results, which are sometime called benchmarks.


1. Identify the task scenarios
This job will be done manually with the functional knowledge.
This job can start as early as design review.

2. Identify the expected results
This job may needs to be done manually with helps of some sort of systems that can hold and sometime generate the data.

The testing automation system should be able to record the expected results.

3. Execute the task flows (candidate for automation)
The QA engineer may perform the task flows once and record the executions.
The executions should be automated as much as possible.
In other word, replaying the tasks back should be possible.  The test execution can be a scheduled job that can be executed frequently.

The expectation to the testing automation system is to allow the jobs to be scheduled and to be monitored.  The results should be recorded with the enough period of time for analyzing the issues.

4. Compare the results with benchmarks (candidate for automation)
This job should be done automatically if possible.
The QA just need to check the exceptions.

The testing automation system should be able to summarize the results and highlight the potential problems

5. Analyze the problems
The QA should review the exceptions and identify if the test needs to be modified, if there is an issue of the system, and who should be responsible for it, and report the problem.








No comments: