Question 5 of 7: Recognising and Correcting a Failing Development Process

Question 5: Recognising and Correcting a Failing Development Process (15 marks)

Question text not reproduced: the examination questions are © Engineers and Geoscientists BC. Open the official past paper (linked at the top of this page) to read the question, then follow the worked solution below.

Part A — The first indications

The useful indications are the ones that appear before the schedule visibly slips, because by the time a milestone is missed the decision that caused it is months old. They fall into three groups.

Quantitative leading indicators. Earned value diverging from actual cost; engineering change requests arriving at a flat or rising rate late in the programme instead of decaying; the open-issue list growing faster than it is closed; requirements still being added after the specification was frozen; prototype build times and first-pass yields not improving between builds; and test-failure counts per build failing to fall.

Behavioural indicators. Reviews that finish early with no substantive challenge; decisions reopened repeatedly without new information; specifications quietly relaxed to match what the current design achieves; a rising proportion of effort spent on status reporting rather than on engineering; key people declining to commit to dates; and the appearance of an unofficial second design being worked on privately because someone has lost confidence in the official one.

Structural indicators. Nobody can state the target specification from memory; the single source of truth has forked into several spreadsheets; manufacturing and service are first consulted after the design is frozen; and the gate reviews have become presentations rather than decisions.

The first group is worth quantifying, because a number changes a conversation that adjectives cannot.

Given. A development programme with a budget at completion of CAD 2,400,000 over 18 months. At the month-9 status date the planned value is CAD 1,200,000, the earned value CAD 900,000 and the actual cost CAD 1,260,000. Engineering change requests over months 4 to 9 have arrived at 6, 7, 8, 9, 10 and 12 per month.

Find. Whether the programme is recoverable on its current trajectory, and what the change data says about design convergence.

1. Compute the cost and schedule performance indices. $$\begin{aligned} CPI&=\frac{EV}{AC}=\frac{900{,}000}{1{,}260{,}000}=0.714\\[2pt] SPI&=\frac{EV}{PV}=\frac{900{,}000}{1{,}200{,}000}=0.750 \end{aligned}$$ The programme has earned 71.4 cents of value per dollar spent and is running at three quarters of planned pace. Extrapolating at the achieved efficiency, $$EAC=\frac{BAC}{CPI}=\frac{2{,}400{,}000}{0.714}=\text{CAD }3{,}360{,}000$$ a forecast overrun of CAD 960,000, or 40 per cent, and a forecast duration of $18/0.750=24$ months.
2. Test whether the plan can still be recovered. The to-complete performance index is the efficiency required over the remaining work to finish at the original budget: $$TCPI=\frac{BAC-EV}{BAC-AC}=\frac{1{,}500{,}000}{1{,}140{,}000}=1.316$$ Against a demonstrated $CPI$ of 0.714 this demands $$\boxed{1.316/0.714=1.84\ \text{times the efficiency the team has ever achieved}}$$ That ratio is the honest signal. A recovery plan asking for a 10 per cent improvement is plausible; one requiring 84 per cent is a request to believe that the previous nine months were anomalous.
3. Test whether the design is converging. In a healthy programme the residual design gap decays geometrically, $g_k=\rho^{k}$ with $\rho<1$, so change requests should thin out as the design settles. Fitting the observed series, $$\rho=\left(\frac{12}{6}\right)^{1/5}=1.149$$ The rate is rising. Projected forward, $\rho^{6}=2.30$: the design is heading twice as far from closure as it was at month 4. For comparison, a converging programme at $\rho=0.62$ reaches a 5 per cent residual in $k=\ln 0.05/\ln 0.62=6.27$, i.e. seven design passes. The change data and the earned-value data are telling the same story from different directions, which is what makes the diagnosis trustworthy.
4. Price the consequence of leaving it alone. Cost of change rises by roughly an order of magnitude per development stage. At the current stage an average change costs about CAD 9,500 against CAD 950 had it been caught one stage earlier, so at 12 changes per month the avoidable exposure is $12\times 8{,}550=\text{CAD }102{,}600$ per month, or CAD 615,600 over the six months to the tooling gate — 64 per cent of the forecast overrun. The overrun is not mysterious; it is late change, priced.

Parts B and C — Five underlying causes, each with a corrective action

Cause 1 — unstable or unvalidated requirements. The specification was never agreed, never made measurable, or continues to change after freeze, so the design chases a moving target and every change ripples. Corrective action: stop design work and re-baseline. Convert every requirement into a metric with a target value and a verification method, obtain explicit sign-off from marketing, manufacturing and service, and put the baseline under change control so that further changes require a named approver and a costed impact assessment. Nothing else on this list can be fixed while the target is moving.

Cause 2 — over-the-wall working: manufacturing, service and suppliers consulted after the fact. Producibility problems are then discovered at tooling, where they cost ten to a hundred times what they would have cost at concept. This is the cause most consistent with the change-request curve above — a rising ρ late in a programme almost always means downstream knowledge arriving late. Corrective action: move to concurrent working with a co-located core team including a manufacturing engineer and a key supplier, and make producibility sign-off a gate condition rather than a downstream review.

Cause 3 — optimistic planning with no contingency and no risk register. The plan assumed first-time success at every step, so the first genuine technical surprise consumes the entire float. Corrective action: re-plan from the achieved productivity, not the original assumption — use the measured CPI and SPI to forecast, maintain a live risk register with owners and quantified exposure, and hold explicit contingency at the programme level rather than hiding it inside individual task estimates.

Cause 4 — unresolved technical risk carried past the gate at which it should have been retired. The hardest unknown is deferred because it is uncomfortable, and the programme optimises the parts it understands while the real risk compounds. Corrective action: identify the highest-uncertainty assumption and test it immediately with the cheapest experiment that can falsify it, ahead of any further detailing. Iterate early and cheaply: the point of a prototype is to fail before the tooling is cut.

Cause 5 — diffuse ownership and ineffective gate reviews. No single person owns the outcome, and gates function as status presentations at which the only available decision is to continue. Corrective action: appoint one accountable programme owner with authority over scope, and give every gate explicit, pre-agreed pass/fail criteria with kill and redirect as real options. A gate that has never stopped a programme is not a gate.