Planning Effective Software Testing (TPDD)

Course Outline

I. The Test Planning Process

Test planning cannot be done in a vacuum. The test plan must integrate smoothly with all the other project plans and consider many variables, both within the testing group and throughout the rest of the organization.

1. Understand how testing fits within the software development lifecycle
2. Understand the role and use of a test plan
3. See how the test plan relates to other plans (e.g. Project plan & Quality plan)
4. List the inputs to test planning
5. List the outputs from test planning
6. Perform peer reviews of the test plan
7. Obtain organizational commitment to the test plan
8. Track progress against the test plan, report status and re-plan

II. Test Plan: Scope and Lifecycle

What is and is not included in the testing activities can be the subject of many disagreements; therefore, the test plan must be explicit about the scope of the testing activities and the testing lifecycle.

1. Identify the requirements against which the testing will be done
2. Define the goals and objectives for testing
3. Identify boundaries - what will and will not be tested
4. Enumerate the phases and steps in the testing lifecycle
5. Identify how the testing lifecycle integrates with the project lifecycle
6. Define specific entry criteria - how you know when testing can begin
7. Define specific exit criteria - how you know when testing is complete
8. Identify testing services that will be purchased rather than done in-house

III. Test Plan:Traceability Matrix

The only way to assure that the test plan covers all of the requirements and goals without unnecessary tests is to have a systematic way to map tests and test cases to those requirements and objectives. Including a traceability matrix with the test plan is the easiest way to satisfy this need.

1. List every requirement and goal or objectives in one place
2. List every test and test case in one place
3. Map requirements to test cases
4. Assure that every requirement has at least one test case
5. Assure that every test case corresponds to at least one requirement
6. Avoid overkill (or under emphasis) in testing
7. Determine the impact of skipping test cases

IV. Test Plan: Required Tests

Before test cases can be identified, the system requirements and testing objectives must be used to compile a list of tests that will be required. This list of tests is the heart of the test plan.

1. Identify tests for functional requirements
2. Identify tests for performance requirements
3. Identify tests for security and safety requirements
4. Identify tests for usability, maintainability and other requirements
5. Define objectives and success criteria for each test
6. Document each test in the traceability matrix
7. Use the traceability matrix to assure complete coverage

V. Test Plan:Test Cases

Actually writing test cases and preparing the related data consumes a significant amount of time. Therefore it is important to estimate and plan for these activities.

1. Enumerate the test cases required to satisfy the objectives for each test
2. Identify positive, negative, boundary and special test cases
3. Define objectives and success criteria for each test case
4. Document each test case in the traceability matrix
5. Use the traceability matrix to assure complete coverage

VI. Test Plan:Test Case Size Estimates

In order to provide a basis for planning the effort, costs and other resources needed for testing, we must estimate the size of each test case and document this in the test plan.

1. Test case description and instructions
2. Input data and/or database records required by the test case
3. Output data and/or database records the test case will produce
4. Special resources required by the test case
5. Execution time for the test case

VII. Test Plan: Resources

Resources to support testing go far beyond just the people who will do the testing. The test plan must account for all required resources.

1. Identify the testing and test case development environment (e.g. hardware, operating systems, networks, software, databases)
2. Specify any special systems (e.g. test automation, defect tracking)
3. Enumerate knowledge and skills needed
4. Plan for hiring, contracting and training

VIII. Test Plan: Effort, Cost, Budget & Schedule

Effort, cost, budget and schedule are usually the items we are asked to provide. But until all of the items in sections II through VII have been identified and estimated, we do not have the information we need to provide these things. These critical parts of the test plan can now be completed.

1. Identify the activities required to produce and execute all of the test cases, track defects, retest and do all of the other tasks associated with the testing lifecycle
2. Estimate the effort required based on the size estimates and identified activities
3. Identify all costs (e.g. labor, equipment, software contracted work)
4. Establish a schedule for all testing-related activities
5. Spread the costs across the schedule to produce a budget
6. Validate budget and schedule against project constraints
7. Resolve budget or schedule issues

IX. Test Plan: Risks

Testing activities have their own unique risks that may not be visible or pertinent to other stakeholders in the project. The testing group should engage in risk management to assure that those items are handled appropriately and included in the test plan.

1. Brainstorm a testing-related risk list
2. Group and consolidate risks
3. Quantify risk probability and impact
4. Make risk tracking plans
5. Make risk mitigation plans
6. Make risk contingency plans

X. Test Plan: Management,Tracking & Reporting

The test plan must identify how the testing group will maintain control over the testing activities and assure that they are progressing as planned. It must also define how they will report status to other project stakeholders and take corrective actions when necessary.

1. Identify measurements that will be used in tracking and managing the testing activities
2. Determine how the data and reports that are generated by the testing process will be stored, managed and made available
3. Determine how often testing status will be checked and who will participate in status checking activities
4. Identify triggers for corrective actions when the testing activities deviate from the plan
5. Determine what must happen when the test plan must be updated
6. Identify all individuals and groups that have a stake in the testing activities
7. Determine how the stakeholders will be involved and kept informed about testing-related activities