Defect Prevention

a key process area for level 5: Optimizing

Defect Prevention involves analyzing defects that were encountered in the past and taking specific actions to prevent the occurrence of those types of defects in the future. The defects may have been identified on other projects as well as in earlier stages or tasks of the current project. Defect prevention activities are also one mechanism for spreading lessons learned between projects.

Trends are analyzed to track the types of defects that have been encountered and to identify defects that are likely to recur. Based on an understanding of the project's defined software process and how it is implemented (as described in the Integrated Software Management and Software Product Engineering key process areas), the root causes of the defects and the implications of the defects for future activities are determined.

Both the project and the organization take specific actions to prevent recurrence of the defects. Some of the organizational actions may be handled as described in the Process Change Management key process area.

Goals

Goal 1

Goal 2

Goal 3

Commitment to perform

Commitment 1 -- The organization follows a written policy for defect prevention activities.

1. Long-term plans and commitments are established for funding, staffing, and other resources for defect prevention.
2. The resources needed are allocated for the defect prevention activities.
3. Defect prevention activities are implemented across the organization to improve the software processes and products.
4. The results of the defect prevention activities are reviewed to ensure the effectiveness of those activities.
5. Management and technical actions identified as a result of the defect prevention activities are addressed.

Commitment 2 -- The project follows a written organizational policy for defect prevention activities.

1. Defect prevention activities are included in each project's software development plan.
2. The resources needed are allocated for the defect prevention activities.
3. Project management and technical actions identified as a result of the defect prevention activities are addressed.

Ability to perform

Ability 1 -- An organization-level team to coordinate defect prevention activities exists.

1. This team is either part of the group responsible for the organization's software process activities (e.g., software engineering process group) or its activities are closely coordinated with that group.

Refer to the Organization Process Focus key process area.

Ability 2 -- A team to coordinate defect prevention activities for the software project exists.

1. This team is closely tied to the team responsible for developing and maintaining the project's defined software process.

Ability 3 -- Adequate resources and funding are provided for defect prevention activities at the project and organization levels.

1. Defect prevention activities are planned into each person's responsibilities, as appropriate.

   ---

   Examples of defect prevention activities include:
   • task kick-off meetings,
   • causal analysis meetings,
   • reviewing and planning proposed actions, and
   • implementing actions.

   ---

2. Management participation in the defect prevention activities is planned.
3. Each software project is represented on the team coordinating defect prevention activities for the organization, as appropriate.
4. Tools to support defect prevention activities are made available.

   ---

   Examples of support tools include:
   • statistical analysis tools, and
   • database systems.

   ---

Ability 4 -- Members of the software engineering group and other software-related groups receive required training to perform their defect prevention activities.

• software quality assurance,
• software configuration management, and
• documentation support.

• defect prevention methods,
• conduct of task kick-off meetings,
• conduct of causal analysis meetings, and
• statistical methods (e.g., cause/effect diagrams and Pareto analysis).

Activities performed

Activity 1 -- The software project develops and maintains a plan for its defect prevention activities.

1. Identifies the defect prevention activities (e.g., task kick-off and causal analysis meetings) that will be held.
2. Specifies the schedule of defect prevention activities.
3. Covers the assigned responsibilities and resources required, including staff and tools.
4. Undergoes peer review.

Activity 2 -- At the beginning of a software task, the members of the team performing the task meet to prepare for the activities of that task and the related defect prevention activities.

1. The software process, standards, procedures, methods, and tools applicable to the task, with an emphasis on recent changes.

   ---

   Changes may be implemented as an experiment to evaluate a recommendation from a previous causal analysis meeting.

   ---

2. The inputs required and available for the task.
3. The outputs to be produced with examples, if available.
4. The methods to be used to evaluate the outputs.
5. The methods to be used to verify adherence to the software process.
6. A list of errors that are commonly made or introduced during the current stage and recommended preventive actions for these errors.
7. The team assignments.
8. The task schedule.
9. The software product quality goals for the task and software project.

Activity 3 -- Causal analysis meetings are conducted according to a documented procedure.

1. Each team that performs a software task conducts causal analysis meetings.
   • A causal analysis meeting is conducted shortly after the task is completed.
   • Meetings are conducted during the software task if and when the number of defects uncovered warrants the additional meetings.
   • Periodic causal analysis meetings are conducted after software products are released to the customer, as appropriate.
   • For software tasks of long duration, periodic in-process defect prevention meetings are conducted, as appropriate.

   ---

   An example of a long duration task is a level-of-effort, customer support task.

   ---

2. The meetings are led by a person trained in conducting causal analysis meetings.
3. Defects are identified and analyzed to determine their root causes.

   ---

   An example of a method to determine root causes is cause/effect diagrams.

   ---

4. The defects are assigned to categories of root causes.

   ---

   Examples of defect root cause categories include:
   • inadequate training,
   • breakdown of communications,
   • not accounting for all details of the problem, and
   • making mistakes in manual procedures (e.g., typing).

   ---

5. Proposed actions to prevent the future occurrence of identified defects and similar defects are developed and documented.

   ---

   Examples of proposed actions include modifications to:
   • the process,
   • training,
   • tools,
   • methods,
   • communications, and
   • software work products.

   ---

6. Common causes of defects are identified and documented.

   ---

   Examples of common causes include:
   • frequent errors made in invoking a certain system function, and
   • frequent errors made in a related group of software units.

   ---

7. The results of the meeting are recorded for use by the organization and other projects.

Activity 4 -- Each of the teams assigned to coordinate defect prevention activities meets on a periodic basis to review and coordinate implementation of action proposals from the causal analysis meetings.

1. Review the output from the causal analysis meetings and select action proposals that will be addressed.
2. Review action proposals that have been assigned to them by other teams coordinating defect prevention activities in the organization and select action proposals that will be addressed.
3. Review actions taken by the other teams in the organization to assess whether these actions can be applied to their activities and processes.
4. Perform a preliminary analysis of the action proposals and set their priorities.

   ---

   Priority is usually nonrigorous and is based on an understanding of:
   • the causes of defects,
   • the implications of not addressing the defects,
   • the cost to implement process improvements to prevent the defects, and
   • the expected impact on software quality.
   An example of a technique used to set priorities for the action proposals is Pareto analysis.

   ---

5. Reassign action proposals to teams at another level in the organization, as appropriate.
6. Document their rationale for decisions and provide the decision and the rationale to the submitters of the action proposals.
7. Assign responsibility for implementing the action items resulting from the action proposals.
   • Implementation of the action items includes making immediate changes to the activities that are within the purview of the team and arranging for other changes.
   • Members of the team usually implement the action items, but, in some cases, the team can arrange for someone else to implement an action item.
8. Review results of defect prevention experiments and take actions to incorporate the results of successful experiments into the rest of the project or organization, as appropriate.

   ---

   Examples of defect prevention experiments include:
   • using a temporarily modified process, and
   • using a new tool.

   ---

9. Track the status of the action proposals and action items.
10. Document software process improvement proposals for the organization's standard software process and the projects' defined software processes as appropriate.

    ---

    The submitters of the action proposal are designated as the submitters of the software process improvement proposals.

    ---

    ---

    Refer to Activity 5 of the Process Change Management key process area for practices covering handling of software process improvement proposals.

    ---

11. Review and verify completed action items before they are closed.
12. Ensure that significant efforts and successes in preventing defects are recognized.

Activity 5 -- Defect prevention data are documented and tracked across the teams coordinating defect prevention activities.

1. Action proposals identified in causal analysis meetings are documented.

   ---

   Examples of data that are in the description of an action proposal include:
   • originator of the action proposal,
   • description of the defect,
   • description of the defect cause,
   • defect cause category,
   • stage when the defect was injected,
   • stage when the defect was identified,
   • description of the action proposal, and
   • action proposal category.

   ---

2. Action items resulting from action proposals are documented.

   ---

   Examples of data that are in the description of an action item include:
   • the person responsible for implementing it,
   • a description of the areas affected by it,
   • the individuals who are to be kept informed of its status,
   • the next date its status will be reviewed,
   • the rationale for key decisions,
   • a description of implementation actions,
   • the time and cost for identifying the defect and correcting it, and
   • the estimated cost of not fixing the defect.

   ---

3. The defect prevention data are managed and controlled.

   ---

   "Managed and controlled" implies that the version of the work product in use at a given time (past or present) is known (i.e., version control), and changes are incorporated in a controlled manner (i.e., change control).

   If a greater degree of control than is implied by "managed and controlled" is desired, the work product can be placed under the full discipline of configuration management, as is described in the Software Configuration Management key process area.

   ---

Activity 6 -- Revisions to the organization's standard software process resulting from defect prevention actions are incorporated according to a documented procedure.

Activity 7 -- Revisions to the project's defined software process resulting from defect prevention actions are incorporated according to a documented procedure.

Activity 8 -- Members of the software engineering group and software-related groups receive feedback on the status and results of the organization's and project's defect prevention activities on a periodic basis.

1. A summary of the major defect categories.
2. The frequency distribution of defects in the major defect categories.
3. Significant innovations and actions taken to address the major defect categories.
4. A summary status of the action proposals and action items.

   ---

   Examples of means to provide this feedback include:
   • electronic bulletin boards,
   • newsletters, and
   • information flow meetings.

   ---

Measurement and analysis

Measurement 1 -- Measurements are made and used to determine the status of the defect prevention activities.

• the costs of defect prevention activities (e.g., holding causal analysis meetings and implementing action items), cumulatively;
• the time and cost for identifying the defects and correcting them, compared to the estimated cost of not correcting the defects;
• profiles measuring the number of action items proposed, open, and completed;
• the number of defects injected in each stage, cumulatively, and over releases of similar products; and
• the number of defects.

Verifying implementation

Verification 1 -- The organization's activities for defect prevention are reviewed with senior management on a periodic basis.

1. A summary of the major defect categories and the frequency distribution of defects in these categories.
2. A summary of the major action categories and the frequency distribution of actions in these categories.
3. Significant actions taken to address the major defect categories.
4. A summary status of the proposed, open, and completed action items.
5. A summary of the effectiveness of and savings attributable to the defect prevention activities.
6. The actual cost of completed defect prevention activities and the projected cost of planned defect prevention activities.

Verification 2 -- The software project's activities for defect prevention are reviewed with the project manager on both a periodic and event-driven basis.

Verification 3 -- The software quality assurance group reviews and/or audits the activities and work products for defect prevention and reports the results.

1. The software engineering managers and technical staff are trained for their defect prevention roles.
2. The task kick-off meetings and causal analysis meetings are properly conducted.
3. The process for reviewing action proposals and implementing action items is followed.

Table of contents Forward one chapter

  antoine@apple2.com 29-04-2000