Designed Experiments — Randomization, Blocking, and a 2³ Factorial: 23-Ind-A5 Quality Planning, Control, and Assurance, May 2017, Q6

Question 6: Designed Experiments — Randomization, Blocking, and a 2³ Factorial (20 marks)

Question text not reproduced: the examination questions are © Engineers and Geoscientists BC. Open the official past paper (linked at the top of this page) to read the question, then follow the worked solution below.

(a) Randomization, blocking, design fractionation, and design resolution

Randomization — assigning experimental runs and material to treatments in random order — is essential because it converts any nuisance variable that changes systematically over time or space (equipment warm-up, operator fatigue, ambient drift) from a source of bias that could masquerade as a real factor effect into ordinary experimental error, whose average effect over all treatments is zero. Without randomization, an unlucky run order can make a purely time-related drift look exactly like the effect of the factor being studied.

Blocking groups experimental units into homogeneous subsets (e.g. one day's raw-material batch, one machine, one shift) so that a KNOWN nuisance source of variation can be removed from the comparison of treatments rather than left to inflate experimental error. Testing the significance of the block effect confirms whether that grouping variable really did contribute meaningful variation; if it is significant, blocking was the right call (it removed real noise, sharpening the test on the factors of interest); if not, little was lost by blocking, but the check itself validates the experimental design choice for future runs.

Design fractionation (running only a carefully chosen FRACTION of the full factorial, a $2^{k-p}$ fractional factorial) has the advantage of dramatically reducing the number of runs needed when many factors must be screened, at the cost of confounding some higher-order interactions with main effects or with each other (aliasing) — effects that are aliased cannot be separately estimated from that one fraction. The design resolution quantifies how severe this trade-off is: Resolution III designs alias main effects with 2-factor interactions, Resolution IV designs alias main effects only with 3-factor-and-higher interactions (2-factor interactions are aliased with each other), and Resolution V designs keep main effects and 2-factor interactions entirely clear of each other, aliasing only with 3-factor-and-higher interactions. Higher resolution costs more runs; the appropriate choice balances how many interactions are realistically expected to matter against the runs that can be afforded.

(b) 2³ factorial: estimating main effects and interaction effects

Given. Three factors (A, B, C), each at two levels ($-$/$+$), full $2^3=8$ treatment combinations, $n=2$ replicates per treatment. Undesirable emission level (ppm):

2³ factorial — undesirable emission level (ppm), two replicates
Treatment	(1)	a	b	ab	c	ac	bc	abc
Rep. 1	32	20	40	42	30	56	58	24
Rep. 2	27	24	35	41	22	46	44	21
Treatment total	59	44	75	83	52	102	102	45

Find. The seven effect estimates (A, B, C, AB, AC, BC, ABC).

Approach. Form each effect's contrast from the eight treatment totals using the standard $\pm1$ sign table for a $2^3$ design in Yates (standard) order, then divide by $2^{k-1}n=4(2)=8$ runs-per-side.

Grand total and grand mean. $$\text{Grand total}=59+44+75+83+52+102+102+45=562,\qquad \bar y=\frac{562}{16}=\boxed{35.13}\ \text{ppm}.$$
Main effects. Contrast$_A=-(1)+a-b+ab-c+ac-bc+abc=-59+44-75+83-52+102-102+45=-14$; dividing by 8: $$A=\boxed{-1.75},\qquad B=\boxed{6.00},\qquad C=\boxed{5.00}.$$
Two-factor interactions. Forming each interaction's sign column as the product of its component main-effect columns and contracting against the totals: $$AB=\boxed{-10.50},\qquad AC=\boxed{0.00},\qquad BC=\boxed{-7.75}.$$
Three-factor interaction. $$ABC=\boxed{-16.25}.$$

Effect	A	B	C	AB	AC	BC	ABC
Estimate (ppm)	−1.75	6.00	5.00	−10.50	0.00	−7.75	−16.25

Concept & Theory

Topic: Randomization, blocking, fractional factorial design and resolution; contrast-based effect estimation in a $2^k$ factorial.

Key relations: Effect $=\dfrac{\text{Contrast}}{2^{k-1}n}$; Contrast $=\sum(\pm1)\times(\text{treatment total})$, signs read off the standard $2^k$ design-matrix column for that effect (interaction columns are the elementwise product of their component main-effect columns).

Why this works: The $\pm1$ sign table is orthogonal by construction, so each contrast isolates exactly one effect's information with zero correlation to any other — this is precisely what lets a SINGLE experiment estimate all seven effects (3 main + 3 two-factor + 1 three-factor) independently from the same 16 observations, rather than needing seven separate one-factor-at-a-time experiments.

Common pitfall: Reading the interaction sign columns from a non-standard run order (the sign pattern must be generated from the FACTOR-LEVEL columns of the actual run, not assumed from a memorized table that may use a different labelling convention).

Source: Montgomery, Introduction to Statistical Quality Control (8th ed.), Ch. 13 (factorial and fractional factorial designs, design resolution, effect estimation).

(c) ANOVA, significance testing at 10%, and the optimal factor-level combination

Given. The effect estimates and treatment totals from part (b); $\alpha=0.10$.

Find. Which effects are statistically significant, and the combination of factor levels giving the lowest mean emission.

Approach. Convert each contrast to a sum of squares, obtain $SS_{Error}$ by subtraction from $SS_{Total}$, form the ANOVA $F$-ratios against $MS_{Error}$, and compare each to $F_{0.10,1,8}$; then read the minimum-emission combination directly off the eight treatment means (safer than combining main effects in isolation once interactions are found significant).

Sum of squares per effect. $SS_{\text{effect}}=\text{Contrast}^2/(2^k n)=\text{Contrast}^2/16$: $$SS_A=12.25,\ SS_B=144.00,\ SS_C=100.00,\ SS_{AB}=441.00,\ SS_{AC}=0.00,\ SS_{BC}=240.25,\ SS_{ABC}=1056.25.$$
Error sum of squares. $SS_{Total}=\sum(y_{ij}-\bar y)^2=2211.75$ (computed directly from all 16 readings); summing the seven effect SS gives $\sum SS_{\text{effect}}=1993.75$, so $$SS_{Error}=2211.75-1993.75=\boxed{218.00},\qquad df_{Error}=16-1-7=8,\qquad MS_{Error}=\frac{218.00}{8}=\boxed{27.25}.$$
$F$-ratios and the 10% critical value. $F_{\text{effect}}=SS_{\text{effect}}/MS_{Error}$ compared against $F_{0.10,1,8}=3.46$: $$F_A=0.45\ (ns),\ \ F_B=5.28\ (p=0.051,\ \boxed{\text{sig.}}),\ \ F_C=3.67\ (p=0.092,\ \boxed{\text{sig.}}),$$ $$F_{AB}=16.18\ (p=0.004,\ \boxed{\text{sig.}}),\ \ F_{AC}=0.00\ (ns),\ \ F_{BC}=8.82\ (p=0.018,\ \boxed{\text{sig.}}),\ \ F_{ABC}=38.76\ (p<0.001,\ \boxed{\text{sig.}}).$$ At $\alpha=0.10$, effects $B$, $C$, $AB$, $BC$ and $ABC$ are statistically significant; $A$ and $AC$ are not.
Optimal combination. Because a significant THREE-factor interaction ($ABC$) is present, main effects cannot be combined additively to predict the best setting; the safe approach is to read the eight treatment means directly: $$(1){=}29.5,\ a{=}22.0,\ b{=}37.5,\ ab{=}41.5,\ c{=}26.0,\ ac{=}51.0,\ bc{=}51.0,\ abc{=}22.5.$$ The minimum is treatment $\boxed{a}$ (A high, B low, C low), mean emission $\boxed{22.0}$ ppm, narrowly ahead of $abc$ (22.5 ppm).

Fig. 6.1 — A×B interaction plot (each point averaged over both levels of C): the two lines are far from parallel and cross in slope, the graphical signature of the significant AB interaction found in the ANOVA.

Quantity	Value
$MS_{Error}$ (8 df)	27.25
Significant effects at $\alpha=0.10$	B, C, AB, BC, ABC
Not significant	A, AC
Lowest-emission combination	treatment $a$ (A+, B−, C−), 22.0 ppm

Check: Although the main effect of A alone is not statistically significant, A cannot be dropped from the recommended setting: A participates in the significant $AB$ and $ABC$ interactions, so its level still materially changes the response depending on B and C — this is exactly why the minimum was located by reading the actual treatment means rather than by combining only the significant MAIN effects.

Concept & Theory

Topic: ANOVA decomposition of a replicated $2^k$ factorial, hypothesis testing on individual effects, and interpreting significant higher-order interactions.

Key relations: $SS_{\text{effect}}=\text{Contrast}^2/(2^kn)$; $SS_{Error}=SS_{Total}-\sum SS_{\text{effect}}$; $F_{\text{effect}}=MS_{\text{effect}}/MS_{Error}$ compared to $F_{\alpha,1,\,2^k(n-1)}$.

Why this works: Each effect has exactly 1 degree of freedom in a $2^k$ design (it is a single contrast), so $MS_{\text{effect}}=SS_{\text{effect}}$ and the $F$-test collapses to a direct ratio against the pooled replication error — the same error that captures the pure within-treatment variability seen directly in the two replicates of each of the 8 cells.

Common pitfall: Picking the "optimal" combination purely from the sign/magnitude of the significant MAIN effects (here that heuristic would suggest B low, C low, ignoring A) when a significant interaction is present — once $AB$ and $ABC$ are significant, the effect of A genuinely depends on the levels of B and C, so only the actual cell means (or a fitted regression model including the interaction terms) give a trustworthy answer.

Source: Montgomery, Introduction to Statistical Quality Control (8th ed.), Ch. 13 (ANOVA for factorial designs, interaction interpretation).

Question 6 of 6: Designed Experiments — Randomization, Blocking, and a 2³ Factorial

Question 6: Designed Experiments — Randomization, Blocking, and a 2³ Factorial (20 marks)

(a) Randomization, blocking, design fractionation, and design resolution

(b) 2³ factorial: estimating main effects and interaction effects

(c) ANOVA, significance testing at 10%, and the optimal factor-level combination