Question 5 of 8: Software Verification and Validation

Question 5: Software Verification and Validation (a) 5, (b) 5, (c) 10 — 20 marks

Question text not reproduced: the examination questions are © Engineers and Geoscientists BC. Open the official past paper (linked at the top of this page) to read the question, then follow the worked solution below.

(a) Verification vs. Validation

Verification asks "are we building the product right?" — it checks that the system conforms to its stated specification, which is a well-defined, largely objective comparison between two documents (the specification and the system's observed behaviour). Validation asks "are we building the right product?" — it checks that the system actually meets the customer's real, underlying needs, which is a comparison against something far less precisely defined than a specification.

Validation is particularly difficult because the specification itself is only ever an imperfect proxy for what the customer actually needs: requirements are gathered from stakeholders who may not fully know or agree on what they want, may change their minds once they see a working system, or may have needs that are genuinely hard to articulate in advance of using the system (a classic finding in requirements engineering is that users often only discover their real requirements by interacting with an early version). Validation therefore cannot be reduced to a mechanical conformance check the way verification can; it requires judgement about fitness for an evolving, incompletely-understood purpose, and its most reliable technique — putting the system in front of real users under real conditions — is exactly the technique that is most expensive and slowest to apply early in a project, when its findings would be cheapest to act on.

(b) Defect-Freedom, Testing and Fitness for Purpose

Complete freedom from defects is neither practically achievable for a non-trivial system nor economically justified: the cost of finding and removing each additional defect rises as defect density falls, while the value of removing a low-impact defect shrinks, so a rational release decision stops testing once the expected cost of the defects still present (probability × consequence) is smaller than the cost of continuing to test for them. What the customer actually needs is not zero defects but a system that is fit for purpose: reliable and safe enough, in the ways that matter for its intended use, to deliver acceptable value.

Testing validates fitness for purpose only up to the coverage and representativeness of the test suite used. A passing test suite demonstrates correct behaviour for the specific inputs and scenarios exercised; it cannot demonstrate correctness for every possible input, since testing (per Dijkstra's well-known observation) can show the presence of defects but never their absence. The closer a test suite's scenarios are to the system's real operational profile, the more confidently a passing result can be read as evidence of fitness for purpose; for systems where that confidence must be very high (safety-critical or highly business-critical systems), testing alone is not considered sufficient and is supplemented with other verification techniques, such as inspection or formal methods, that reason about the program rather than merely sampling its behaviour.

(c) Checklist of Errors Not Caught by a Compiler

A compiler enforces language syntax and (for statically-typed languages such as C/C++/Java) static type rules, but it cannot catch errors that are syntactically and often type-correct yet semantically wrong. A program-inspection checklist for such errors, applicable across C/C++/Java-family languages, includes:

• Uninitialized variables used before being assigned a value.
• Off-by-one errors in loop bounds or array indexing (e.g. iterating <= array length instead of <).
• Mismatched or unchecked array bounds, reading or writing past the end of a buffer.
• Assignment used where comparison was intended (= instead of == inside a conditional) — legal and type-correct in C/C++ when the assigned expression is numeric.
• Incorrect operator-precedence assumptions, e.g. mixing bitwise and relational operators without parentheses.
• Memory leaks: allocated memory that is never freed on some or all code paths.
• Dangling pointers / use-after-free: a pointer used after the memory it referenced has been freed.
• Integer overflow, particularly in a computation whose intermediate result exceeds the type's range even though the final result would not.
• Division by zero on a denominator whose value is not provably non-zero at that point.
• Ignored return values or error codes from a function call whose failure case is never checked.
• Resource leaks: file handles, sockets or locks acquired but not released on every exit path (including exception paths).
• Race conditions in code shared between threads without adequate synchronization.
• Overly broad or swallowed exception handling that catches and silently discards an exception that should propagate or be logged.
• Floating-point equality comparisons (== on floating-point values) that are almost always wrong due to representation error.
• Logic errors in boolean conditions, e.g. && used where || was intended, or an inverted condition.
• Missing break in a switch statement, causing unintended fall-through.