What is a software regression?

A software is not working as how it works before.

All bug should be reviewed and the regressions should be identified.

Identifying and fixing regression issues is important as it involves the creditability of the company and the products and it generates frustration to the users of the software.

A development team that generates the software regression may have process or quality issues.

Development managers should monitor the trend of the regressions reported.
Developers should analyze the root cause of the regression.  Which product feature changes or which bug fixes caused the software regression.

If a software is shipped with software regression, it means that there may be a test case missing is the regression testing.

Regression Analysis and Resolution Lifecycle

  • Customer file a service request
  • Customer support files a bug
  • While the developer is analyzing the issue, if the defect is confirmed and the developer suspects that there is a software regression, flag the bug with the regression status as 'Candidate'
  • If the problem exists before in the GA version and the current version, it is not a regression, the regression status should be set as 'Not a regression'  It is a problem not previous caught.
  • After further analysis, and the bug or code change that causes the regression is identified, the regression status should be set to 'Confirmed'.  The original bug number(s) should be recorded. A comment should be given to describe why a regression happens.  "Lack of training', 'Missing in the design', 'Process Improvement', etc. 


Regression test related lifecycle

  • The regression test status should be recorded to indicate whether a test should be created.
  • After the regression test is created, the regression test can be closed from the QA perspective.

This is the close loop analysis from QA perspective.





Software Sustain Engineering

Sustain Engineering is not necessary a required function within the software development organization.

Initially when I first become a software engineer, there is no such function.  Sustain engineering is part of the duties of the regular software development groups.  Sustain Engineering gradually become a separate function.

The sustain engineering team is handling the products that have moved into production.  The sustain engineering team monitors the product throughout the rest of the lifecycle until the product terminated and becomes obsolete.

Sometime people refer the sustain engineering team as the maintenance team.

The nature of the work could be different.  The work tends to be more ad hoc and more issue-oriented.  In other word, the every day life of the members in the sustain engineering team is dissimilar to the core development team which is more project based, with the scheduled activities.    It could be rewarding too.  The sustain engineering may see different challenges and may have a bigger coverage comparing to the software engineers that are assigned in the core development team.  Within a year, the member of the sustain engineering  team may touch different part of the products while the members in the core development team only work on one or two projects within a specialized functional area or technologies.







What is a deferred bug?

A deferred bug is a bug that will not be fixed in the current release.

A typical reason why a bug is deferred since fixing it does not give us significant returns than the investment (ROI).  The deferral decision is a business decision and should not be made by the developers themselves.

Deferring a bug means that we leave a known issue to the product and we do not fix it. 

Typically we have the following justifications:
  • It is a corner case.
  • The particular usage scenario is not common.
  • There is a workaround available.
  • We will provide a patch to fix this problem immediately after the release.

The following are excuses but cannot justify unless we can one of the above reasons:
  • It is too late to be fixed in the development cycle of a release.
  • It will introduce significant delay of the release.

One of the important considerations other than just the delay of the release is the risk to the release.

Fixing a bug may introduce other bugs.
Fixing a bug is not just a development effort.  The test may need to be executed again.

The risk may not be just a schedule risk, but a quality risk.

Accept 360

This is a software for requirement management.

We use it for project management.




Release Exit Criteria

This is a check list to show the release readiness.

This is similar to the phase gate in the project lifecycle.

Some people also define the "Entry Criteria" to indicate when a phase can start.

Here are some samples:
  • Functional Design and Technical Design Specifications have been reviewed and signed off by stake holders.
  • User Experience Review and Signed off
  • Coding  Complete
  • Testing Complete
  • Translation (i18n)
  • Documentation is ready

The coding work can be categorized as follows:
  • The coding for the new key features as described in the release content.
    • UI - Front End
    • Back End Process
    • Model / Database 
    • Batch Process
    • API
  • Forward Porting of the bugs fixed in the prior release by the sustaining engineering team
  • Platform Porting (if supporting multiple platforms) 
  • Upgrade / Data Migration



During the course of the project, the work can be measured by percent complete.
Sometime the percent complete method is too subjective.  A alternate way is to break the work into smaller units and measure three status : started? work in process? completed?  and assigned, 0%, 50%, and 100%.  The total percentage can be a rolled up based on the weighted average.


Testing (coverage) are complete

Here are some of the typically testing categories:
  • Installation (for new customers)
  • Functional Test (For new features)
  • Integration / Interops
  • Regression Test (for existing, old features)
  • Performance Test
  • Stress Test
  • Platform Certifications
  • Upgrade (for existing customers)
  • Security Test
  • Accessibility Test
To measure the testing coverage, we can use percentage. Testing execution can be similar to the coding work. The measurement can also be done by smaller units of work.

The overall progress of the QA testing cycle can be measured by the backlog of the bugs to be fixed in the queue.  Which can be further converted # of days of the backlog based on the fixing rate.  For examples, typically 2 bugs fixed per person each day and five resource in total.  The 100 bugs can be fixed in 50 person days.  It means that we have 10 days of backlog.

The incoming rates can also be a good leading indicator.  Initially the bugs logged per day may be high.  After the testing has been done for a while, the number of bugs logged per day may be reduced.

Deferral rate is a good measurement. A exit criteria can be the deferral rate is less than 10%.  The deferral rate can be further broken down by priories (P1, P2, P3, etc).  Deferral means that the bugs found is the testing cycle is not being fixed. Deferral rate is too high means that the issues are passed to the future releases.  It also means that the list of known issues are high.



A release is more than just engineering and testing. Other than the above typical development work.  Here are some of the release exit criteria
  • Export Compliance
  • Security Compliance
  • Third Party software license
  • Product's bill of materials

We may need to consider other activities as part of the release readiness:
  • Demo Env and build the demo scenarios
  • Field Readiness - Sales and Marketing
  • Analyst Briefing
  • Press Release
  • Partner training
  • Support training
  • Hand Over to Sustain Engineering


Agile Process - Different Types of Meetings

Daily Work Day SCRUM briefing

  • Please describe what you did
  • If you are blocked, explain and get the team to help


Bi-Weekly SPRING meeting

  • Close the tasks for the prior SPRING cycle
  • Retroactive: What are the lessons learned and what can be improved
  • Add and Edit tasks for the upcoming SPRING cycle
  • Assign Tasks to owners

Running Application on Cloud (1)

Running Apps on Cloud is not new.

The mainframe apps were running as hosted apps.  At that time, hardware business is also software business. The users were using terminals to access their apps.

We later moved to the client/server model. The desktop computing ago come.  Two key technologies enabled this client/server computing: Wintel PC and LAN. The benefit is to empower the end users. 
Later, we found that client / server is a maintenance nightmare.  It is when the client / server model was modified to the "thin client" or to the three tie model.

Web based computing come about the same time.  We then moved the thin client part to web.  The degree of using browser varies.  Gradually people found out putting logic into browser is not a bad idea.  That's where Web2.0 kicked in.

Now, we are in another paradigm swift.  The key driving technology is the network bandwidth is reaching a point to let us to run the apps over Internet. At least, we now know better about mastering it.  Of course, there are computing technologies as well, such as virturalization.






Unit Testing

Unit testing is the test executed by developers.
Unit testing is part of coding.  I learned from day one when my mentor told me at that time when I start my career in software development.
Coding is not done if unit testing is not done.

Unit testing may involve writing codes. We write codes for testing our codes.

In the modern software development tool, unit testing codes can be generated and we can simply run it after coding work is done. It is not always the case, however. 

Unit testing cannot replace system testing.  Unit testing is to test a small unit of the software.  While the unit A and the unit B both pass unit testing, unit A and B may not work together.

Unit testing is important since when we test features or test the integration, the unit testing error may becomes the bottleneck. It introduces higher costs and the delay to the system testing schedule.  A developer should feel shame if an issue is found in the system testing when it can be found in unit testing.

The challenge of unit testing typically comes from the the lack of the testing env. The unit may rely on other units.

Developers do not do proper unit testing are typically not because they do not like to, it is typically due to the overhead on setting up the env.

A unit testing env should be provided to ease the unit test work.



Regression testing

Regression testing is to ensure the software still working as designed.

Since we continue making the changes to software for introducing new features or fixing bugs, the changes may accidentally introduce bugs which cause the other existing features that were working become not working.

Regression testing is the way to detect this type of issues.

Regression testing should be automated if possible.

Ideally it should be executed every day.

The problem needs to be uncovered as early as possible.

The test cases used in regression testing are typically not new test cases.  They are the test cases used in feature testing.

Obviously the test case can become invalid since the product feature may change.  The test case should be associated with the product feature somehow so when the product functionality changes, we remember to up date the regression test.

Here is a good article. - Regression Testing with Regression Testing Tools and Best Practices.  I agree every single word described.

Team Work

Some people are good at finding issues.
Some people are good at solving issues.

Sometimes those people who are good at finding issues cannot show their values as the issues they found are ignored or overlooked by the organization.

Sometimes they are afraid of raising issues and telling others.

Some people good at finding issues but do not know how to express themselves or do not know how to properly express themselves at the right time and in a right way.  They may get some recognitions but won't get the credits.

Sometimes those people who are capable to solve the issues are not solving issues for some reasons, such as lack of the motivation or wrongly assigned to different tasks.

There are people who are lacking self awareness or organization awareness.
There are good observers who can monitor organization and find the above mismatch.

There are action takers who know the above organizational issues and can act on it and fix the organizational issue.

We call those people managers.




From Installation to Provisioning

I learned the concept of provisioning within the past five year.  I do not remember when I first heard of this term. I do not think that I heard the provisioning process in enterprise apps development before.
The provisioning seems a concept borrowed from the hardware business.

I do not know the exact meaning, but based on my understanding, the idea is to enable the various components to work together.

It involves of installing, configuring, deploying, initializing, and removing application components.

This is part of the overall IT automation.  It is a special offering from an enterprise software vendor.

When you buy software individually from different vendors, you have to spend time to make sure that they can work together.  Earlier, various vendors need to work together to collaborate the integration processes and some 3rd party professional services need to participate to make software packages from different vendors work together.

A simple example is to think of the problem in a client server apps.  You install the database and you create users in the database.   You then install the client and configure the connection to the database. Since the database and the client come from different vendors, the installations are separate.  The process of creating user and entering the user in the client is your responsibility.  The vendor providing the client software will need to test and certify the database they are using and document how you can use their software with the database.

With the provisioning process, the installation, creating the user, entering the connection become one process.  You can still enter or override a user, but only once.  The provisioning process does it for you.

This is the power of consolidation and integration.

It is the value the enterprise software vendor can provide with its special position.  They acquire companies so they can provide the integration out of the box for the IT department.

From business perspective, it extends the product and service they are offering to the business used to be taken by IT or by 3rd party service provider.

From software development perspectives, the line of codes we are providing are, if using the client server apps example, database software, the client software, and the provisioning scripts.  It may not be a big deal for the architecture involving only two components.  It is a big deal if the software architecture involves many components.  



How to measure the product success?

You may have multiple product lines.How you measure their success?
You can have a simple metrics to measure how much revenue they are bringing into the company.

The way how we measure and evaluate the performance can change the behaviors.
By treating them as individually units, you are encouraging them to complete.

Don't get me wrong. The resource is limited. Comparing them so you can properly allocating resources to them.  They do compete against each others on resources.

Competition is not necessary a bad thing.  However, it may not be always a good thing, either.

One way we can do is to measure how much it can bring the revenues to other products.

In a big company, there may be a mix of product lines or product units.  For those established products with an established installed base, their revenue should not be measured alone.  They should be measured by how much revenue that can bring by upselling the other products.  Only with this measurement, we can guarantee that there will be enough incentive for building an integrated solution.


What is a software bug?

A bug is a defect of the software product.
A defect is about something is not working as designed.

Some people are confusing a bug with a functional gap.

A functional gap is something that was not designed properly.

The design of the product is not working as expected.

This differentiation is useful as the process of  handling a bug could be different from handling a functional gap.

It also shows the ownership of the problem and how the problem should be handled and can be avoided in the future.

From the end user and from the customer's perspective, a bug and a functional gap may not make that different.  For people working as part of the software development organization, correctly categorizing the issue is important.




Project Manager, Development Manager, and Work breakdown structure

A person who is good at the execution will know the importance of creating work breakdown structure (WBS).

A good executor makes things happen, or get things done (GTD).  A good executor not just come up idea and plans, but know "how to" make thing happen.  They know what the WBS looks like.

Coming up a WBS, from PMI's PMBOK perspective, is a planning task. It is a function performed by a project manager.

However, I found that coming up a work breakdown structure is actually sometime a technical job that requires people to have certain level of the domain knowledge.  Similar projects have similar WBS, especially at the high level.structure.  When I go further down to the detail level, I found that the work breakdown structure will be more specific and requires people to have the knowledge to come up the breakdown and come up the list of tasks.

A project manager is really a role the assigned resource plays.  A project manager is really not a separate function. I feel that we cannot easily make a good project manager who can manage construction project well to be a project manager for software development.  There are shared knowledge that can be useful for any project manager.  The project manager to manage a specific type of project need to know that work down to certain level to do the project manager job.

Who should create the WBS? Can the job of creating WBS to be assigned to someone else? 

I think that it cannot be totally avoid anyway when we go down to the detail levels.  For a new project that has not yet been done before and no similar project the project manager can use to come up the WBS, the project manager need  to work with the architect who need to come up the high level components to help project manager to manage the project.

I think that this process also define how the architect works with the development manager.  I think that typically the person with the development manager title in software development organization should play the project manager role.  Sometimes, a project leader or an architect may also play the project manager role. A person who never played the project manager can still be a good architect, but is never qualified to be a development manager.




GTD Contexts

1. Someday/Maybe List

This is a Wish List.  If I have time I should spend some time on this task.
I still review it regularly to remind myself that I do not forget them.

This is different from the list that I will be regularly once a week on a specific scheduled time.

This is the list of tasks that I am not working on it.  The items listed here are unplanned lists.

It could be a project that still has not yet been broken down.

It can be a project idea.  They can be removed after researched and planned and the individual breakdown items ( not the whole item) are moved to other context category.

This a blog posting about Seven Problems with a Someday/Maybe List.

2. Special Occasions

I am assigning tasks to future of me under some occasions.

Typically these are part of my regular life.  Either once a week or  may be a particular time everyday.

The typical example used are @Home, @Work, @Errands, or Reading List.

I feel that it still very important to talk with people in person even we can send emails and using messaging.  Having one on one with your boss and your directs are part of the system developed within my company.

My contexts are organized by people
@Boss 1on1
@Mary 1on1
@HQ Office
@Pleasanton Office

@Weekly Staff Meeting
@Weekly IDC Meeting
@Home with kid
@This weekend

The more context you have, the more regular scheduled time blocks you have.  It may become less flexible, but I feel that I will be more organized.

This is a good article that talks about GTD contexts -  What is (not) a GTD context?




Learn GTD

1. What is GTD



2. How to create a to-do list?



Summary

Three categories:
  • INBOX: Today: Review multiple times a day
  • WEEKLY: Week: Review it daily
  • MAYBE: Longer: Review it at least once a week

Use Wikidpad (3): Publish HTML pages

Although Wikidpad is personal wiki tool and does not really allow multiple users to update the wiki pages, Wikidpad can be a useful tool for generating HTML pages.