Question 3 of 7: Early Indications of a Failing Design Process, and Recovery

Question 3: Early Indications of a Failing Design Process, and Recovery (15 marks)

Question text not reproduced: the examination questions are © Engineers and Geoscientists BC. Open the official past paper (linked at the top of this page) to read the question, then follow the worked solution below.

The three indications below are chosen because each is early, measurable and leading rather than lagging. A missed milestone is none of these: by the time a milestone is missed the damage is already done, and the useful signals are the ones that predict the miss weeks or months in advance.

Part A — Indication 1: schedule and cost performance indices below unity and diverging

Given. A development programme with a budget at completion of CAD 12.0 M over 18 months. At the month-9 status date the planned value is CAD 6.0 M, the earned value CAD 4.5 M, and the actual cost CAD 5.4 M.

Find. Whether the programme is in trouble, and what it will cost at completion if nothing changes.

1. Form the two performance indices. The schedule and cost performance indices are $$\begin{aligned} \text{SPI}&=\frac{EV}{PV}=\frac{4.5}{6.0}=0.750 \\ \text{CPI}&=\frac{EV}{AC}=\frac{4.5}{5.4}=0.833 \end{aligned}$$ so the team is producing three-quarters of the work it planned, and paying CAD 1.20 for every dollar of work it does produce. Either index below unity is a warning; both below unity and still falling is the signature of a design process that is not converging.
2. Project the outcome. The optimistic estimate at completion assumes the cost overrun continues but the schedule recovers, $$\text{EAC}_{\text{cost}}=\frac{\text{BAC}}{\text{CPI}} =\frac{12.0}{0.833}=\text{CAD }14.4\ \text{M},$$ while the estimate that assumes both indices persist — which is what the empirical evidence supports once a programme is a third of the way through — is $$\text{EAC}=AC+\frac{\text{BAC}-EV}{\text{CPI}\times\text{SPI}} =5.4+\frac{7.5}{0.625}=\boxed{\text{CAD }17.4\ \text{M}}$$ a 45 % overrun. The to-complete performance index needed to still finish on budget is $$\text{TCPI}=\frac{\text{BAC}-EV}{\text{BAC}-AC}=\frac{7.5}{6.6}=1.136,$$ i.e. the team must suddenly become 36 % more efficient than it has ever been. That is the diagnostic: when the TCPI required exceeds the CPI achieved by more than about 10 %, the plan is no longer credible.

Part A — Indication 2: requirements churn that does not decay

A healthy design process shows requirement changes falling steeply after the specification is baselined, because uncertainty is being retired. A failing one shows a constant or rising rate, which means the team is still discovering what the product is supposed to do while it is designing it. Given. Requirements changing at 8 % of the baselined set per month, sustained for six months. Find. The cumulative churn. Since each month's changes apply to the already-changed set, $$\text{cumulative factor}=(1+0.08)^{6}=1.587 \;\Longrightarrow\;\boxed{58.7\ \%\ \text{cumulative change}}$$ Nearly six requirements in ten have moved since the baseline. Any design work completed at the start of that window has a better than even chance of being invalid, and no amount of engineering effort will converge against a target moving at that rate.

Part A — Indication 3: an open-issue backlog whose arrival rate exceeds its closure rate

Given. 84 open design issues today; new issues arriving at 26 per week and being closed at 19 per week; the design freeze is nine weeks away. Find. The backlog at freeze, and the closure rate that would be needed to clear it.

1. Project the backlog. The net accumulation is $26-19=7$ issues per week, so $$B_{\text{freeze}}=84+(26-19)\times9=\boxed{147\ \text{open issues at freeze}}$$ against 84 today. The backlog is not merely large, it is growing, which is the part that matters: a large but shrinking backlog is a programme in recovery.
2. Compute the capacity actually required. To reach zero at freeze the team must close the existing backlog as well as everything that arrives, $$\dot{c}_{\text{req}}=26+\frac{84}{9}=35.3\ \text{per week},$$ which is $35.3/19=1.86$ times the current closure rate, an 86 % increase in issue-resolution capacity. Stating it that way converts a vague sense that the team is struggling into a resourcing decision with a number attached.

Part B — Rectifying each of the three

Rectifying indication 1 (adverse and diverging performance indices). The first action is diagnostic, not corrective: decompose the variance by work package, because a programme-level CPI of 0.833 is an average that usually conceals one or two packages performing far worse and the rest performing acceptably. Adding resource uniformly is the standard wrong answer, and on a design task it is frequently counter-productive because the added people consume the time of the people who already understand the problem. Once the failing packages are identified, three levers exist and should be applied in this order: re-scope — move requirements that are not needed for first delivery into a follow-on release, which is the only lever that reduces the work rather than redistributing it; re-plan — rebaseline against a schedule the team can actually meet, since a plan nobody believes stops functioning as a control instrument entirely; and re-staff selectively, adding experienced people to the specific failing packages, ideally by moving them from packages that are ahead. In parallel, tighten the reporting interval so the response to the next divergence is measured in weeks rather than months, and escalate honestly — the EAC of CAD 17.4 M is a governance decision, not an engineering one, and concealing it removes the only people who can authorise a change of scope.

Rectifying indication 2 (requirements churn). Churn is almost always a symptom of an incomplete front end rather than of an undisciplined customer, so the remedy has to address the cause before it addresses the rate. Convene a short, intense requirements re-validation with the actual decision-makers on the customer side and close the open questions that are generating the changes — typically a handful of unresolved use cases or an unstated regulatory position. Then re-baseline once, formally, and place the new baseline under a change-control board with the authority to reject changes, not merely to record them. Every change request from that point states its cost, schedule and technical impact before it is considered, which by itself removes most low-value requests. Where the requirement genuinely cannot be settled — a market that has not decided, a standard still in draft — the right response is architectural rather than procedural: isolate the volatile requirement behind a defined interface so that changes to it do not propagate, and defer its commitment as long as the architecture allows. Finally, track the churn rate as a reported metric with a target decay: a rate that is not measured is not managed.

Rectifying indication 3 (growing issue backlog). Begin by triaging the whole backlog into three classes — issues that must be closed before freeze, issues that can be carried into the next phase with an agreed disposition, and issues that on inspection are duplicates or are no longer valid. On a backlog of 147 this routinely removes a quarter of the items for no engineering effort at all, and it converts an undifferentiated pile into a work list. Attack the arrival rate as well as the closure rate: a persistent 26 issues per week usually traces to a small number of unstable interfaces or immature subsystems, and stabilising those reduces arrivals far more cheaply than closing their consequences one at a time. Add reviewing capacity where the queue is, which is usually at the approval step rather than at the engineering step, and run a fixed daily triage so that no issue waits for a weekly meeting. If, after all of that, the required closure rate remains near the 86 % increase computed above, the honest conclusion is that the freeze date is wrong and must be moved — freezing a design over 147 open issues does not remove them, it converts them into production concessions and warranty claims at ten times the cost.