19-Soft-A6 Software Quality Assurance, May 2015, Q3

Question 3 (10 marks)

Question text not reproduced: the examination questions are © Engineers and Geoscientists BC. Open the official past paper (linked at the top of this page) to read the question, then follow the worked solution below.

Part (a) — main testing objectives (Glen Myers's classic framing, adopted by Pressman): testing is a process of executing a program with the intent of finding errors — not of showing the program works. A good test case is one with a high probability of finding an as-yet-undiscovered error, and a successful test is one that uncovers such an error. Testing cannot prove the absence of defects, only their presence; it increases confidence that the software does what it is intended to do, within the bounds actually exercised.

Part (b) — three principles for designing effective test cases.

All test cases should be traceable to requirements. Every requirement should have at least one test case that would fail if that requirement were not met, so the most severe defects (from the customer's point of view) are the ones caught first.
Tests should be planned long before testing begins. Test-case design can start as soon as the requirements model is complete (or even during it), which finds specification defects earlier and avoids designing tests around implementation quirks.
The Pareto principle applies to testing. Roughly 80% of the errors uncovered during testing will likely be traceable to about 20% of the program modules; test-case design should isolate and heavily target those suspect modules once they're identified.

Part (c) — characteristics of testability (things that make software easy to test):

Operability — the better it works, the more efficiently it can be tested (a system with many bugs makes it hard to isolate new ones).
Observability — what you see (outputs, states, error messages, logs) is what you get; distinct output for each input, and internal state is inspectable.
Controllability — the degree to which software (and hardware) states can be controlled directly by the test engineer, so inputs/conditions can be set precisely.
Decomposability — the software is built from independent modules that can be tested independently, containing the scope of any one test.
Simplicity — functional, structural and code simplicity reduce the number of test cases needed and make defects easier to isolate.
Stability — few changes are requested during testing, and changes are controlled (a moving target is far harder to test).
Understandability — clear design/documentation means testers can derive better test cases and interpret failures faster.

Topic: Testing objectives, effective test-case design principles, and testability.

Key relations: objective = maximize probability of finding defects, not prove correctness; principles = traceability + early planning + Pareto targeting; testability = operability + observability + controllability + decomposability + simplicity + stability + understandability.

Why this works: if the goal is to find defects (not prove absence), the most efficient tests are the ones aimed at requirements and at the historically defect-prone 20% of modules, and that efficiency is only realizable if the software itself is testable (you can control its inputs and observe its outputs).

Common pitfall: designing tests only after the code is complete and only around "does it run" — this misses the requirement-traceability principle and tends to test the easy paths rather than the defect-prone ones the Pareto principle predicts.

Source: Pressman, Software Engineering: A Practitioner's Approach, 9th ed., Ch. 17 §17.2 (A Strategic Approach to Testing) and Ch. 18 §18.1–18.2 (Testing Strategies / Testability).

Question 3 of 8

Question 3 (10 marks)