Bug Bash

Bug Bash is referring to an event that everyone focuses on finding and fixing bugs, but nothing else.  It is an indication of the software having a severe quality issue and cannot be taken care by the normal process.

Having a bug bash may mean that QA needs helps as they did not uncover bugs while they should.  However, in another angle, it may mean that during the normal process, QA was not given enough resource or time to run the test.  The defects of products may be accumulated over time.   The problem is that the investment on QA may not be linear comparing to the growth of development.  When we count the scenarios to be covered, when the software is getting complex, it will grow very fast while the development resources including maintenance does not need to grow as fast.

Slowing down and giving QA more authority as a gate keeper  may be sometime more important than having a bug bash.

Test Failure and Test Blocker

Test Blocker

Blocking QA from testing may just require a small code fix or may just need a fix to the QA env configuration.

It affects the QA testing and affects the QA testing schedule and productivity.  Unblocking QA is critical and needs to be treated as an urgent issue.

Notes and Tips about Jira

New product or project development:

1. Create a new JIRA project

You need to have the project administration permission to do so.
In an organization, this permission will not be granted to many people.

You can give it a key.  The key will be prefix of issues number for issues created in the project.
You can give it a unique icon, call Avatar.

A JIRA project is sometime mapped to "Product" within software organizations.  A new product may be a "project", but when you have multiple releases and start maintenance cycles, you are developing and maintaining a product which is typically created as a JIRA project.

2. Create a Board

For a new development project, a new broad can be created for an existing JIRA project, or for a JQL, which is a query for collecting issues from multiple projects.

This is a very good way to run an "integration" project, when a project involves the teams that are from multiple product and from multiple organizations.

You can create multiple boards.  Multiple boards can exist simultaneously.  A board is actually just a view of the issues in different columns.

Pickup a ticket from the backup

It is nice that a developer to pick up a prioritized ticket from JIRA without anyone assigning the ticket to him or her.

However, we also need to think about impacts of this approach.

Best Practice: Use JIRA for tracking your tests

Zephyr for JIRA is a nice tool for doing this job.

Bug Filing and Closing by QA

Here are the notes from today. Some useful guidelines:


When we file a bug, we need to ensure that there is an instruction of how to reproduce the issue.

We should refer the bug to a test case.

--

When we close a bug, we should also ensure that the issue is described properly in the bug.  This is including how to verify it.

If we do not already have a test case, we should consider creating one.  This is for regression testing to ensure the issue won't come back again.

In the end, all bugs, regardless that they are filed by QA or not will be covered in QA test cases.

--

No need to worry about too many test cases.  We still need to prioritize them.  Not all of them will be tested over again.

Also, some of them can be automated, while others not.  It is not a criteria for not creating a test case based on whether it can be automated.

Oracle Bug Status Codes and the Process

Found the bug status codes used by Oracle:

http://www.oracle.com/us/corporate/acquisitions/hyperion/bugstatusdescription-072226.pdf

I think that it is a sophisticated system and represents accumulated knowledge over time at Oracle.

Here are the processes I followed to use this system.

Handling bugs in scrum process

This is an interesting topic about how to manage and prioritize the maintenance efforts in the scrum process.

Cloud Operator Acceptance Testing before deploying into cloud

The software package is developed in the dev env.  The dev env does not necessary reassemble the real production env.   It supposes to be a simple, isolated env for dev to code and perform unit testig.  That's why we need the QA env.  The QA env should cover the testing cases that represent the customer's deployments.

However, a cloud env may have additional requirements.

No regression bugs

How to ensure that there is no regression bugs while introducing new features?

Here are my thoughts:

1. Knowledge Sharing

Developers need to have the knowledge about the whole product, especially about how the feature he or she is responsible is being used.

No one would intend to make the code broken unless for political reasons, such as avoiding dependencies.  We can argue that the developers are careless.  However, in my experience, most of the cases, lake of the knowledge is the reason.

2. Document the dependencies

Identify test plan availability for each bug

It was a good practice followed by QA in the company I worked before.

It is enforced by the QA manager but also known by several Dev Managers.  It is for close loop analysis.

Identify testing scenarios as part of defining the requirement

When defining the requirement, it is also the time to define the acceptance criteria.  The requirement is about "I want the system to be able to do...".   When I started writing requirements, I also try to start with the sentence "You should be able to ...".

For example, the requirement is "You should be able use the same password to login to BI as the password used to login to Apps".  The testing scenario will be:

Differences between a main release and a patchset

A product feature that requires the full cycle of planning, designing, test planning, test execution, packaging, documentation, etc. should be put into the main releases.

A patch or a patchset that are release vehicles for delivering the fixes to the products and we should avoid to put product features into it.  It will makes the cycle of of delivering patches becomes longer and will actually introduce uncertainties to the patch.

A developer who want to put the features into a patch typically does not mean to create any harms.  They have eagers to contribute to the product and should not be discouraged.

We  just need to help people to get familiar with the development methodologies.

That is why the in-job training is important for the quality of the products.

Automated Testing - what to be automated?

Automated Testing helps improving QA's productivity.  It automates the jobs that, otherwise, need to be done manually.

What are the jobs to be automated?

What are QA's jobs?

QA identifies the tasks that the users of the software will do.  The QA executes the tasks with the assumptions that the software will be used under certain circumstances .  The QA checks if the tasks can be done and whether the results are expected.  Before QA performs the tasks, the QA identifies the expected results, which are sometime called benchmarks.

Foolproof or Poka-Yoke in software development

I learned this concept when I studied Quality Control in college.

This is considered part of the TQM (Total Quality Management) practices.  Each employee should contribute to design a foolproof system which can ensure that there is no mistake can happen.

The idea is that we do not rely on the individuals to be careful when they do not need to be careful.  The system should prevent or reduce the likelihood of the error from happening.

A foolproof system  takes out the human errors from the equation.

--

I think that the concept can be applied to software development as well.

QA Test Plan Review and Sign Off

QA should come up the test plan that describes what will be tested and when the test execution will be started and will be done.

The Dev team should review and sign off the test plan.

DHQA - Development Handoff to QA

The Development Handoff to QA (DHQA) is an important event in a software project.

It is the event to indicate that the code is complete and ready for testing.

It means that developers finish the coding and the dev side of testing, including unit testing and certain level of integration testing.

Here are some of the check list items:

Software Testing for Cloud Applications (1)

Testing software for cloud is a little bit different from the testing for the software that will be deployed on premise.

For software that are delivered as both cloud service and the independent software, there are two layers of testing.  The software is first tested by the software QA and then tested by the team who test the software for cloud.

What is a deferred bug?

A deferred bug is a bug that will not be fixed in the current release.

A typical reason why a bug is deferred since fixing it does not give us significant returns than the investment (ROI).  The deferral decision is a business decision and should not be made by the developers themselves.

Deferring a bug means that we leave a known issue to the product and we do not fix it.

Typically we have the following justifications:

Unit Testing

Unit testing is the software testing executed by developers themselves.
Unit testing is part of coding.  I learned from day one when my mentor told me at that time when I start my career in software development.
Coding is not done if unit testing is not done.