Question 5 of 10: Software Testing

Question 5: Software Testing (a) 5, (b) 5, (c) 10 — 20 marks

Question text not reproduced: the examination questions are © Engineers and Geoscientists BC. Open the official past paper (linked at the top of this page) to read the question, then follow the worked solution below.

(a) Why Testing Only Shows the Presence of Errors

A test suite exercises the program on a finite, chosen subset of its (typically astronomically large or infinite) input space. If any test in that subset fails, the program is provably incorrect — a single failing test is a valid proof of a defect. But if every chosen test passes, that only shows the program behaves correctly on the inputs actually tried; it says nothing about the inputs that were never tried, any one of which could still trigger a defect. Since it is generally infeasible to test every possible input (or every possible execution path/state), passing all tests can never be used to conclude the program is defect-free — only that no defect has yet been found by this particular finite sample.

(b) Regression Testing, and Why Automation Simplifies It

Regression testing is re-running a system's existing test suite after a change (a bug fix, a new feature, a refactor) specifically to check that the change has not broken any previously working behaviour — that the system has not "regressed." It is necessary because software components are rarely as independent as their interfaces suggest: a change made for one purpose can have unintended side effects on unrelated parts of the system through shared state, a changed data format, or a subtle timing dependency, and the only reliable way to catch such a side effect is to re-verify everything that was already known to work, not just the area that was deliberately changed.

Run entirely by hand, regression testing is expensive in direct proportion to how large the existing test suite has grown, which creates a perverse incentive to skip it, run only a hand-picked subset, or run it rarely — exactly the situations in which a regression is most likely to slip through. An automated test suite (each test coded so it runs and checks its own expected result without a human operator) plus a testing framework (the harness that discovers, executes, and reports every test's pass/fail status in one run) removes that cost: the full suite can be re-run after every single change, in minutes, with no incremental human effort per run, which is precisely why continuous-integration pipelines re-run the whole regression suite on every commit rather than only before a release. Automation also makes regression testing repeatable in the strict sense — the same test executes the same steps and applies the same pass/fail criterion every time, removing the variability of a human tester re-deriving "does this still look right" by eye on each run, which both catches subtler regressions and makes a failure immediately attributable to the specific change that introduced it.

(c) Test Cases

1. Integer array sort routine

2. Non-blank character counter

3. YYYY/MM/DD date-validation module