22-Mec-B5 Product Design and Development · May 2018

Question 5 of 7: Designing a New Product Against Refining an Existing One

Nivaar worked solution (AI-drafted; not reviewed by a licensed engineer)

Notes on this paper

Paper format. National Exams, May 2018, 16-Mec-B5 Product Design and Development — THREE (3) hours, OPEN BOOK, one approved Casio or Sharp calculator permitted. Question 1 is compulsory and carries 40 marks; four of the remaining six questions are chosen, each worth 15 marks, for 100 marks in total. Only the first five questions appearing in the answer book are marked. Most answers are expected in essay form or as tables, figures and charts, and clarity and organisation carry marks in their own right.

Scope of this solution. All seven questions are answered in full, not the five a candidate would attempt, so that the paper works as a study resource. Where the examiner offers a choice of product, one is selected and carried consistently through every part, which is exactly what the question's own guidance note asks for. Numeric illustrations are engineering estimates built from stated, ordinary data; every one of them.

Reference texts for 22-Mec-B5.

Ulrich, K. T. and Eppinger, S. D., Product Design and Development, 6th ed., McGraw-Hill — the framework text for this exam code (development process, concept selection, product architecture, DFM, economics).
Dieter, G. E. and Schmidt, L. C., Engineering Design, 5th ed., McGraw-Hill — problem definition, requirement cascades, material and process selection, design for X.
Pahl, G., Beitz, W., Feldhusen, J. and Grote, K.-H., Engineering Design: A Systematic Approach, 3rd ed., Springer — systematic embodiment design and the function structure.
Boothroyd, G., Dewhurst, P. and Knight, W., Product Design for Manufacture and Assembly, 3rd ed., CRC Press — the DFA index and process-based cost estimating.
Ashby, M. F., Materials Selection in Mechanical Design, 5th ed., Butterworth-Heinemann — material indices, screening and ranking, process-cost models.
Kalpakjian, S. and Schmid, S. R., Manufacturing Engineering and Technology, 7th ed., Pearson — process capability and manufacturing constraints on design.
Cooper, R. G., Winning at New Products, 5th ed., Basic Books — stage-gate governance and gate criteria.
Vaver, D., Intellectual Property Law: Copyright, Patents, Trade-marks, 2nd ed., Irwin Law, together with the Canadian Intellectual Property Office (CIPO) guides — the Canadian statutory frame for Question 4.

Question 5: Designing a New Product Against Refining an Existing One (15 marks)

Question text not reproduced: the examination questions are © Engineers and Geoscientists BC. Open the official past paper (linked at the top of this page) to read the question, then follow the worked solution below.

Part A — Comparing the two thought processes (6 marks)

The two designers are solving different kinds of problem, and almost every difference in how they think follows from that.

The clean-sheet designer faces an ill-defined problem. Nothing is settled: not the function, not the architecture, not the user, sometimes not even whether the need is real. The first work is therefore problem definition rather than solution — establishing that the need exists, framing it as a function rather than as a device, and deliberately holding the solution space open. Reasoning is by first principles and by analogy to adjacent industries, because there is no incumbent to reason from. Uncertainty is high and, importantly, is expected to be high, so the plan is built around resolving it: prototypes are built to answer questions rather than to demonstrate progress, and the highest-risk assumption is tested first. Large iteration is normal, whole concepts are abandoned, and the architecture — how function is allocated to physical chunks and where the interfaces sit — is the single decision with the longest shadow, because it constrains everything the product can become for a decade.

The refining designer faces a well-defined problem inside a fixed frame. The architecture exists and is not up for negotiation; the baseline is a running product with real cost data, real warranty data and real customers; and the change must land in a specific gap between what the product does and what it should do. The reasoning is differential rather than absolute: the question is never "how should a blender work" but "what does this change do to the parts of the design it touches, and what else does it touch". The constraints that dominate are the ones the clean-sheet designer does not have — carry-over parts and their tooling, amortisation not yet recovered, service-parts commonality, homologation and certification that must not be re-opened, plant layout, and the supplier agreements already signed.

The decisive psychological difference is the asymmetry of risk. For the clean-sheet designer, the downside of a failed concept is a concept discarded early and cheaply; the real risk is being too timid to find the good architecture. For the refining designer, the downside is a regression — breaking something that worked for customers who already own the product — and that is worse than a missed improvement, because it converts satisfied customers into complaining ones. That asymmetry is why refinement work is dominated by change control, impact analysis and regression testing, and why a refining designer who behaves like a clean-sheet designer is dangerous.

Part B — Sources of customer data, and the challenges (5 marks)

Where the clean-sheet designer turns. There is no user base, so the data must come from analogues and from observation rather than from asking. Lead users — people who already have the need acutely and have improvised their own solutions — are the highest-yield source, because their workarounds are a specification written in physical form. Ethnographic observation of the task in its real setting reveals what people do rather than what they say they do. Analogous markets show how a related need has been solved elsewhere. Latent-need interviews, concept clinics with rough prototypes, and Kano classification of attributes into must-be, one-dimensional and delighter categories complete the set. The output is qualitative and its purpose is discovery, so the right sample-size question is not statistical precision but saturation.

Given. Each user interview independently uncovers any given latent need with probability $p = 0.10$.

Find. The number of interviews required to uncover 90 per cent and 95 per cent of the needs, and the marginal yield of one further interview.

Model coverage and solve for the sample size. A given need is missed by all $n$ interviews with probability $(1-p)^{n}$, so coverage is $C(n)=1-(1-p)^{n}$ and $$n=\frac{\ln(1-C)}{\ln(1-p)}=\frac{\ln(0.10)}{\ln(0.90)}=21.9 \Longrightarrow \boxed{n=22\ \text{interviews for 90 per cent}}$$ and $\ln(0.05)/\ln(0.90)=28.4$, so 29 interviews for 95 per cent.
Check the marginal value, which is what actually stops the work. Coverage at 22 interviews is 90.15 per cent and at 23 it is 91.14 per cent, so the twenty-third interview adds $\boxed{0.98\ \text{percentage points}}$. This is the quantitative form of the saturation rule of thumb that twenty to thirty interviews are enough: it is not that the population is small, it is that the return per interview has collapsed.

Needs coverage against interview count. The curve is steep to about fifteen interviews and flat after twenty-five, which is why the practical guidance is a range rather than a formula.

Where the refining designer turns. The refining designer has the data source the other one lacks: the installed base. Warranty claims coded by part and failure mode, service and repair records, returns and their stated reasons, dealer and call-centre reports, telematics or connected-product duty-cycle data, customer-satisfaction surveys tracked over model years, competitive teardowns, and controlled A/B comparisons where the product allows it. This data is abundant and quantitative, which permits statistical inference rather than saturation judgements.

Given. A satisfaction survey is to estimate a proportion to within $\pm 4$ percentage points at 95 per cent confidence, with no prior estimate of the proportion.

Find. The required sample size.

Size the survey at the worst-case proportion. For a proportion, $n = z^{2}p(1-p)/E^{2}$, maximised at $p=0.5$, so $$n=\frac{(1.96)^{2}(0.5)(0.5)}{(0.04)^{2}}=600.25 \Longrightarrow \boxed{n=601\ \text{respondents}}$$ Note the contrast with Part B's first calculation: 22 interviews for discovery, 601 responses for measurement. Confusing the two — running 601 surveys to discover needs, or 22 interviews to measure a proportion — is the commonest methodological error in this area.

The challenges, which differ in kind. For the clean-sheet designer: users cannot articulate latent needs, and asking them directly produces a request for a faster horse; there is a persistent say-do gap between stated preference and observed behaviour; lead users are unrepresentative of the mainstream by construction, so their needs must be filtered rather than adopted; and small qualitative samples invite over-generalisation from a vivid single case. For the refining designer: the data is severely selection-biased, because warranty and complaint data comes only from customers who bothered, and silent dissatisfaction — the customer who simply buys another brand — is invisible; failure codes are entered by technicians paid to close jobs quickly and are therefore noisy; satisfaction scores are ordinal but are routinely averaged as though they were interval; and the loudest segment of the installed base drives the agenda toward incremental fixes for existing customers while the reasons non-customers did not buy go unmeasured. Both designers share one hazard: confirmation bias in interpretation, which is why the interpretation should be committed to in writing before the data is examined.

Part C — How each assesses the success of the final design (4 marks)

The clean-sheet designer must answer a question about the whole product: did it solve the need, and did it earn its place? The instruments are adoption against the plan; realised gross margin against the business case; the net present value actually achieved against the one that justified the investment; retention and repeat purchase, which distinguish a product people bought from a product people use; and the engineering verification that every requirement was met. There is a trap here worth naming: a clean-sheet product that meets every specification and sells nothing has failed, so verification is necessary and not sufficient. The design must also be judged on whether the architecture it established supports the next three products, because that is the asset the programme really bought.

The refining designer must answer a much narrower question: did the specific metric move, and did anything else break? The instruments are correspondingly paired. A directed measurement on the changed attribute, and a regression suite on everything carried over.

Given. A design change is intended to reduce a specific failure mode. Baseline warranty incidence is 42 claims per 1,000 units at 12 months in service on 8,000 units; after the change it is 31 per 1,000 on 8,000 units.

Find. Whether the improvement is statistically real.

Test the difference of two proportions. With $p_1 = 0.042$, $p_2 = 0.031$ and $n_1 = n_2 = 8{,}000$, the pooled proportion is $\bar p = 0.0365$ and the standard error of the difference is $$SE=\sqrt{\bar p(1-\bar p)\left(\frac{1}{n_1}+\frac{1}{n_2}\right)}=\sqrt{0.0365(0.9635)(2/8000)}=0.002965$$ so the test statistic is $$z=\frac{p_1-p_2}{SE}=\frac{0.011}{0.002965}=\boxed{3.71}$$ which corresponds to a two-sided probability of about 0.0002. The improvement is real, not sampling noise, and the refining designer can now claim it.

The remaining instruments follow the same discipline: process capability on any characteristic the change touched, cost variance against the business case for the change, and — the one that decides whether the work should have been done at all — whether the field data twelve months later shows the intended improvement without a compensating rise anywhere else. A refinement that moves its own metric while degrading a neighbour has not succeeded; it has moved the problem.

Topic. Clean-sheet against incremental design; voice-of-the-customer methods; sample sizing for discovery and for measurement; verification of an improvement.

Key relations. Needs coverage, $C(n)=1-(1-p)^{n}$, inverted as $n=\ln(1-C)/\ln(1-p)$; survey sample size for a proportion, $n=z^{2}p(1-p)/E^{2}$; two-proportion test, $z=(p_1-p_2)/\sqrt{\bar p(1-\bar p)(1/n_1+1/n_2)}$.

Why this works. The coverage model treats each interview as an independent Bernoulli trial against each latent need, so the probability a need survives undetected falls geometrically and coverage rises with diminishing returns; that geometric decay, not any property of the population, is what produces the familiar twenty-to-thirty-interview guidance. The survey formula comes from the normal approximation to the binomial with the variance evaluated at its maximum, $p=0.5$, which makes the answer conservative when the true proportion is unknown. The two-proportion test pools the samples under the null hypothesis that the change did nothing, so a large $z$ is evidence against exactly that hypothesis.

Common pitfall. Using a measurement tool for a discovery task. A 601-respondent survey can tell you what fraction of owners are dissatisfied with a feature that already exists; it cannot tell you about a need nobody has words for, because the question was never on the form. Conversely, drawing a percentage from twenty-two interviews and quoting it as a market statistic gives a confidence interval wide enough to be useless. The second pitfall is declaring a warranty improvement from a raw drop in claim counts without a significance test, when incidence at low rates is noisy enough to move that far by chance on small fleets.

Source. Ulrich and Eppinger, Product Design and Development, 6th ed., Ch. 5 (identifying customer needs) and Ch. 9 (concept testing); Dieter and Schmidt, Engineering Design, 5th ed., Ch. 3 and Ch. 12; Montgomery, Introduction to Statistical Quality Control, 7th ed., Ch. 4 (inference on proportions).

← Question 4 Next: Question 6 →