Question 4 of 8: Software Testing

Question 4: Software Testing (a) 5, (b) 5, (c) 5, (d) 5 — 20 marks

Question text not reproduced: the examination questions are © Engineers and Geoscientists BC. Open the official past paper (linked at the top of this page) to read the question, then follow the worked solution below.

The answers below reuse that verified content directly.

(a) Why Testing Only Shows the Presence of Errors

A test suite exercises the program on a finite, chosen subset of its (typically astronomically large or infinite) input space. If any test in that subset fails, the program is provably incorrect — a single failing test is a valid proof of a defect. But if every chosen test passes, that only shows the program behaves correctly on the inputs actually tried; it says nothing about the inputs that were never tried, any one of which could still trigger a defect. Since it is generally infeasible to test every possible input (or every possible execution path/state), passing all tests can never be used to conclude the program is defect-free — only that no defect has yet been found by this particular finite sample.

(b) Black-Box vs. White-Box Testing

Black-box (functional/specification-based) testing derives test cases purely from the component's specified behaviour — its inputs, outputs and stated requirements — with no reference to how it is implemented internally. White-box (structural) testing derives test cases from the component's internal structure, aiming to exercise specific statements, branches, or paths through the code. Black-box testing's strengths are that it can be designed as soon as the specification exists (even before the code is written), that it is independent of implementation details and so survives a refactor of the internals unchanged, and that it directly checks whether the software does what it is supposed to do. Its weakness is that it cannot guarantee any particular code coverage — a path through the code may simply never be exercised by any test derived purely from the spec, and defects arising from unusual internal states invisible at the interface can be missed. White-box testing's strength is the reverse: it can systematically guarantee coverage of statements, branches or paths, catching internal logic errors, dead code, and off-by-one conditions that a black-box view would never reveal. Its weakness is that it is tied to a specific implementation (a refactor with unchanged behaviour still requires new white-box tests), it requires access to source and specialised coverage tooling, and passing every internal path proves nothing about whether the code actually implements the intended requirement — a consistently-wrong implementation can have 100% white-box coverage and still fail every black-box case.

(c) Developer Testing vs. Independent Testing

For developers testing their own code: they have the deepest knowledge of the implementation, so they can efficiently target the boundary conditions and internal branches most likely to be wrong, and testing while writing (unit-level, incremental) catches defects at the cheapest possible point — before the code is ever integrated. It also builds a habit of designing for testability from the start.

Against: a developer's mental model of what the code "should do" is exactly the model that produced the bug in the first place — if a developer misunderstood a requirement, their own tests will typically encode the same misunderstanding and so will not catch it. Developers can also unconsciously avoid stressing the parts of the code they are least confident in, and there is a natural incentive (even if unintentional) to write tests the code is known to pass rather than tests designed to break it. An independent test team, working from the specification rather than the implementation, brings a different, adversarial perspective and is not blind to the same assumptions.

The two are not mutually exclusive: the standard practice (and the one implied by asking for arguments "for and against" rather than a single answer) is that developers perform unit testing of their own components as they are written, while an independent team performs integration, system, and acceptance testing against the requirements — each catches classes of defect the other is structurally poor at catching.

(d) Black-Box Test Cases

1. Integer array sort routine

2. Non-blank character counter