23-Ind-A6 Systems Simulation · May 2018

Question 9 of 9: 2 3 Screening Experiment for Warehouse Throughput

Nivaar worked solution (AI-drafted; not reviewed by a licensed engineer)

Notes on this paper

National Exams — May 2018 — 17-Ind-A6 Systems Simulation. Three-hour, closed-book exam; one of two permitted calculators (Sharp or Casio), one 8.5″×11.0″ aid sheet (both sides). Format: three sections — Section A (four concept questions, candidates choose any two, 10 marks each, 20 marks total), Section B (three methods questions built around one continuing warehouse-simulation case study, candidates choose any two, 15 marks each, 30 marks total), Section C (two applications questions, candidates choose any one, 20 marks each). All nine questions are solved below for completeness. Two source anomalies are flagged where they occur: the front-page summary table states Section A is "Do 2 of 3," while Section A's own instructions and its four printed question sets read "two of the following four" — the printed four-question section is answered in full here; and Part C Question 2's sub-parts (a)–(c) are never printed anywhere in the paper, even though the results text for (d)–(f) explicitly refers back to "the factorial design matrix in (a)." Two-page Normal-distribution tables were supplied with the exam; the values below are the same table values obtained by direct computation.

Reference texts: Banks, Carson, Nelson & Nicol, Discrete-Event System Simulation (5th ed., Pearson) — random-number/random-variate generation, input modeling and goodness-of-fit testing, output analysis (warm-up, replication length, batch means vs. replication/deletion), and comparing alternative systems; Montgomery, Design and Analysis of Experiments (current ed., Wiley) — single-factor ANOVA, multiple comparisons, and 2^k factorial designs with interaction analysis (Part C).

Question 9 (Part C.2): 2³ Screening Experiment for Warehouse Throughput (20 marks)

Question text not reproduced: the examination questions are © Engineers and Geoscientists BC. Open the official past paper (linked at the top of this page) to read the question, then follow the worked solution below.

Check — missing sub-parts (a)–(c) Sub-parts (a), (b), and (c) are never printed anywhere in the exam — the text jumps straight from the factor list to results that reference "the factorial design matrix in (a)." The unique way to lay out three two-level factors across the stated eight runs is the full $2^3$ factorial in standard order, so that design is reconstructed below and answered as part (a); parts (b)/(c) cannot be reconstructed and are not attempted. The results table also names the experimenter "Ms. Strange" where every other question in this paper uses "Ms. Case" — reproduced verbatim as a second, minor source inconsistency.

Given. Three two-level factors — A: relief practice (Mass$=-$, Tag$=+$); B: machine availability (3 ASRS$=-$, 4 ASRS$=+$); C: staffing (30 FTE$=-$, 35 FTE$=+$) — run as a full $2^3$ factorial, $5$ replications per run ($N=40$); Grand Sum $=160{,}975$, pooled sample variance $=88.86$:

Run results, 5 replications each
Run	Rep1	Rep2	Rep3	Rep4	Rep5	Sum	Avg	Var
1	4014	4016	4011	4015	4018	20074	4014.80	6.70
2	4021	4014	4012	4014	4013	20074	4014.80	12.70
3	4014	4016	4017	4018	4015	20080	4016.00	2.50
4	4017	4014	4014	4018	4015	20078	4015.60	3.30
5	4033	4034	4034	4034	4032	20167	4033.40	0.80
6	4034	4032	4032	4032	4036	20166	4033.20	3.20
7	4031	4031	4031	4034	4032	20159	4031.80	1.70
8	4034	4037	4034	4035	4037	20177	4035.40	2.30

(Check: the source's own printed "Avg" column is arithmetically inconsistent with its own "Sum" column in every row — e.g. Run 1 prints Sum $=20074$ but Avg $=4024.38$, while $20074/5=4014.80$; Run 8 prints Sum $=20177$, Avg $=4044.40$, while $20177/5=4035.40$. The Avg values shown above are recomputed as Sum$/5$ rather than the source's own printed figures. The Sum and Var columns, and the Grand Sum/variance given for the whole-sample ANOVA, are all mutually consistent (the Grand Sum $160{,}975$ equals the sum of the eight Sum values exactly) and are what the analysis below uses throughout.)

Find. (a, reconstructed) the design matrix; (d) which factors are significant by ANOVA at $\alpha=0.05$; (e) evidence of interaction; (f) the overall best factor setting.

Approach. Lay out the standard-order $2^3$ design; partition the between-run sum of squares into seven orthogonal single-degree-of-freedom contrasts (three main effects, three two-way interactions, one three-way interaction) using the run sums, and test each against the pooled within-run error.

(a, reconstructed) Design matrix. Standard (Yates) order for the three two-level factors:

2³ full-factorial design matrix
Run	A: Relief	B: Machines	C: Staffing
1	$-$ (Mass)	$-$ (3 ASRS)	$-$ (30 FTE)
2	$+$ (Tag)	$-$ (3 ASRS)	$-$ (30 FTE)
3	$-$ (Mass)	$+$ (4 ASRS)	$-$ (30 FTE)
4	$+$ (Tag)	$+$ (4 ASRS)	$-$ (30 FTE)
5	$-$ (Mass)	$-$ (3 ASRS)	$+$ (35 FTE)
6	$+$ (Tag)	$-$ (3 ASRS)	$+$ (35 FTE)
7	$-$ (Mass)	$+$ (4 ASRS)	$+$ (35 FTE)
8	$+$ (Tag)	$+$ (4 ASRS)	$+$ (35 FTE)

Within-run and total sums of squares. $$SSW=4\sum_{run}\text{var}=4(6.70+12.70+2.50+3.30+0.80+3.20+1.70+2.30)=4(33.2)=\boxed{132.8}$$ $$SST=(N-1)(88.86)=39\times88.86=\boxed{3465.54},\qquad SSB_{runs}=SST-SSW=\boxed{3332.74}$$

Partition $SSB_{runs}$ into the seven $2^3$ contrasts. For each effect, contrast $=$ (sum of run-sums at the $+$ level) $-$ (sum at the $-$ level), and $SS=(\text{contrast})^2/N$:

2³ factorial ANOVA
Effect	Estimate	SS	F	Significant at $\alpha=0.05$?
A (Relief)	0.75	5.63	1.36	No
B (Machines)	0.65	4.23	1.02	No
C (Staffing)	18.15	3294.23	793.79	Yes
AB	0.85	7.23	1.74	No
AC	0.95	9.03	2.17	No
BC	−0.35	1.23	0.30	No
ABC	1.05	11.03	2.66	No

with error $df=8(5-1)=32$, $MSW=132.8/32=4.15$, and critical value $F_{0.05,1,32}=4.15$. Only $C$ clears the threshold: $$\boxed{\text{Only factor C (staffing level) is significant at } \alpha=0.05}$$

(e) Interaction. None of $AB$, $AC$, $BC$, $ABC$ clears $F_{crit}=4.15$ (the largest, $ABC$, reaches only $2.66$), so there is no statistical evidence of interaction between any pair — or all three — of the factors: throughput responds to staffing level on its own, without that effect depending on the setting of relief practice or machine count.
(f) Best overall setting. Since only $C$ is significant, with a POSITIVE effect ($+18.15$, i.e. the higher level raises throughput), the higher staffing level should be used: $$\boxed{C=35\text{ FTEs (statistically required)};\ A,\ B: \text{either level (no detectable effect on throughput)}}$$ Relief practice and machine availability show no statistically detectable effect at this sample size, so — on throughput grounds alone — either level of each is equally defensible; a practical tie-break is to prefer whichever level is cheaper to operate (e.g. 3 ASRS units over 4, since $B$ is not significant) unless a non-throughput consideration favours the other level.

Item	Result
(a) design	full $2^3$ factorial, 8 runs, standard order
(d) significant factors	C (staffing) only, $F=793.79\gg4.15$
(e) interactions	none significant (max $F_{ABC}=2.66<4.15$)
(f) best setting	C = 35 FTE; A, B either level

Concept & Theory

Topic. $2^k$ full-factorial screening design, effect estimation via contrasts, and factorial ANOVA.

Key relations. effect $=\bar y(+)-\bar y(-)$; single-df $SS=(\text{contrast})^2/N$; $SST=\sum SS_{\text{effects}}+SSW$.

Why this works. Because a $2^k$ design is fully orthogonal, the between-run variability decomposes EXACTLY (no shared variance) into $2^k-1$ independent single-degree-of-freedom effects, so every main effect and every interaction is tested with its own dedicated $F$-ratio against the same pooled within-run error — no separate experiment is needed per factor.

Common pitfall. Reading a numerically nonzero interaction effect (e.g. $ABC=1.05$) as "there IS an interaction" without running its own $F$-test against the error term — with only 5 replications per run, small nonzero sample effects are expected from pure noise alone, and $F_{crit}=4.15$ is exactly the threshold that separates real from apparent effects here.

Source. Montgomery, Design and Analysis of Experiments, ch. 6 ($2^k$ factorial designs, effect estimation, ANOVA for factorial designs).

← Question 8 Back to the paper →

Question 9 of 9: 2 3 Screening Experiment for Warehouse Throughput

Question 9 (Part C.2): 23 Screening Experiment for Warehouse Throughput (20 marks)

Question 9 (Part C.2): 2³ Screening Experiment for Warehouse Throughput (20 marks)