23-Ind-A6 Systems Simulation · Undated paper

Question 10 of 10: 2 3 Full Factorial Screening Experiment for Process Throughput

Nivaar worked solution (AI-drafted; not reviewed by a licensed engineer)

Notes on this paper

National Exams May 2019 — 17-Ind-A6, Systems Simulation, 3 hours duration. Section A: do 3 of 5 (30 marks); Section B: do 1 of 2 per the front-page summary table — the Part B section instructions on the page itself read “do 2 of 3” instead, a genuine inconsistency in the source paper's own front matter (flagged below); Section C: do 1 of 2 (20 marks). This study resource answers all ten questions across the three parts.

Check — front-page/section instructions disagree. The cover page's summary table states “Section B: Do 1 of 2 Questions, Total marks: 20,” but Part B's own section banner (page 4) reads “Complete two of the following three sets of questions,” and Part B in fact contains three questions worth 15 marks each. The cover page's own overall total (70 marks across “5 Questions”) is likewise only consistent with the “2 of 3” reading (30+30+... does not match 70 either way exactly, since 30(A)+2×15(B)+20(C)=80, one section choice short of 70); this is an internal inconsistency in the exam's own printed materials. All three Part B questions are answered in full below regardless.

Reference texts. Banks, Carson, Nelson & Nicol, Discrete-Event System Simulation (5th ed.) — primary text for random-variate generation, input/output analysis, variance reduction, verification & validation, and design of simulation experiments.

Question 10 (Part C.2): 2³ Full Factorial Screening Experiment for Process Throughput (20 marks)

Question text not reproduced: the examination questions are © Engineers and Geoscientists BC. Open the official past paper (linked at the top of this page) to read the question, then follow the worked solution below.

Given. Three two-level factors: $A$ = WIP level ($5=-,\ 10=+$), $B$ = queue discipline (FIFO$=-$, LIFO$=+$), $C$ = arrival process (Scheduled$=-$, Random$=+$); $5$ replications of each of the $2^3=8$ runs; Grand Sum $=575$; sample variance of all $40$ observations $=5.266$.

Five-replication factorial data
Run	A	B	C	Rep 1	Rep 2	Rep 3	Rep 4	Rep 5	Sum	Var
1	−	−	−	14	16	11	15	18	74	6.7
2	+	−	−	21	14	12	14	13	74	12.7
3	−	+	−	14	16	17	18	15	80	2.5
4	+	+	−	17	14	14	18	15	78	3.3
5	−	−	+	13	14	14	14	12	67	0.8
6	+	−	+	14	12	12	12	16	66	3.2
7	−	+	+	11	11	11	14	12	59	1.7
8	+	+	+	14	17	14	15	17	77	2.3

Find. (a) the $2^3$ design and how to execute it; (b) which main effects/interactions are significant at $\alpha=0.05$, and whether any interaction is present; (c) the practical impact of the results on the search for a throughput-minimizing (search-time-minimizing) configuration.

Approach. Lay out the standard 8-run $2^3$ design in Yates (standard) order for (a); compute each effect's contrast from the run sums, convert to a sum of squares, and compare each to the pooled within-run error via $F$-tests for (b); interpret practically for (c).

(a) Design the $2^3$ experiment. With $3$ two-level factors there are $2^3=8$ distinct treatment combinations. Laid out in standard (Yates) order, coding each factor's low/high level as $-/+$:
2³ design matrix, standard order
Run A (WIP) B (Queue) C (Arrival)

1 5 FIFO Scheduled

2 10 FIFO Scheduled

3 5 LIFO Scheduled

4 10 LIFO Scheduled

5 5 FIFO Random

6 10 FIFO Random

7 5 LIFO Random

8 10 LIFO Random

Process: (i) fix the $8$ treatment combinations above; (ii) randomize the run order to avoid confounding any trend over time (e.g. equipment drift) with a factor effect; (iii) replicate every run enough times (here $5$) to estimate the within-run error variance; (iv) record throughput for every run/replication; (v) analyze via the $2^3$ contrast method — compute each main-effect and interaction contrast from the run totals, convert to sums of squares, and test each against the pooled error mean square in an ANOVA.

2³ design matrix, standard order
Run	A (WIP)	B (Queue)	C (Arrival)
1	5	FIFO	Scheduled
2	10	FIFO	Scheduled
3	5	LIFO	Scheduled
4	10	LIFO	Scheduled
5	5	FIFO	Random
6	10	FIFO	Random
7	5	LIFO	Random
8	10	LIFO	Random

(b) ANOVA — effect contrasts and significance test. Using $\pm1$ signs read off the design matrix, each effect's contrast is $\sum(\pm1)\times(\text{run sum})$, and its sum of squares is $SS=\text{contrast}^2/(n\cdot2^k)$ with $n=5$ replications, $2^k=8$:

Contrasts and sums of squares
Effect	Contrast	SS
A (WIP)	15	5.625
B (Queue)	13	4.225
C (Arrival)	−37	34.225
AB	17	7.225
AC	19	9.025
BC	−7	1.225
ABC	21	11.025

Error: with the given total sample variance $5.266$ over $N-1=39$ degrees of freedom, $SS_{total}=39\times5.266=205.4$ (this also equals $\sum(\text{run sum}^2/n)$ minus the grand-total correction, confirming internal consistency with the Grand Sum of $575$). Summing the $7$ effect sums of squares, $SS_{treatments}=72.6$, so $$SS_{error}=SS_{total}-SS_{treatments}=205.4-72.6=132.8,\qquad df_{error}=N-8=40-8=32,\qquad MS_{error}=132.8/32=4.15.$$

2³ ANOVA at α = 0.05 (F_crit(1,32) = 4.15)
Effect	SS	df	MS	F	Significant?
A (WIP)	5.625	1	5.625	1.36	No
B (Queue)	4.225	1	4.225	1.02	No
C (Arrival)	34.225	1	34.225	8.25	Yes
AB	7.225	1	7.225	1.74	No
AC	9.025	1	9.025	2.18	No
BC	1.225	1	1.225	0.30	No
ABC	11.025	1	11.025	2.66	No
Error	132.8	32	4.15	–	–

Only Factor $C$ (arrival process) clears $F_{0.05,1,32}=4.15$ ($F_C=8.25$, $p=0.007$): $$\boxed{\text{Arrival process (C) is the only significant main effect; WIP level and queue discipline are not, and no two- or three-way interaction is significant}}$$

(c) Practical impact on the search for a good (search-time-minimizing) solution. Because $C$ is the only significant effect and its contrast is negative ($-37$), moving from Scheduled ($-$) to Random ($+$) arrival decreases the response by $2\times37/(5\times4)=3.7$ units on average across the other factor settings — so the low-response (better, if smaller is the goal here) setting is Random arrival. With WIP level and queue discipline both non-significant, and with no significant interactions, the search for a good configuration can be simplified sharply: those two factors can be set on whichever level is operationally convenient (e.g. cheapest to implement) without materially affecting the response, and further experimental effort should concentrate on confirming and exploiting the arrival-process effect (and possibly adding new candidate factors) rather than continuing to explore the WIP/queue-discipline design space, which this experiment shows is close to flat.

Item	Result
(a) design	8-run $2^3$ full factorial, standard order, randomized, 5 replications for error estimation
(b) significant effect(s)	Arrival process (C) only, $F=8.25>F_{crit}=4.15$
(b) interactions	none significant (all interaction $F
(c) implication	fix WIP/queue discipline for convenience; exploit the Random-arrival setting to reduce the response

Concept & Theory

Topic. $2^k$ full factorial design of experiments, contrast-based effect estimation, and ANOVA significance testing with replication.

Key relations. Contrast $=\sum(\pm1)\times(\text{run total})$; effect $=\text{contrast}/(n\cdot2^{k-1})$; $SS_{effect}=\text{contrast}^2/(n\cdot2^k)$; $SS_{error}=SS_{total}-\sum SS_{effects}$, $df_{error}=N-2^k$.

Why this works. The orthogonal $\pm1$ coding of a full factorial design lets every main effect and every interaction be estimated independently from the same $8$-run matrix (no factor's estimate is confounded with another's), and replicating each run supplies a genuine, model-free estimate of pure error against which every effect can be tested.

Common pitfall. Concluding “no significant effects” means the factors don't matter at all, rather than that this particular experiment (at these particular levels, with this much replication) couldn't distinguish WIP/queue-discipline effects from noise; and forgetting to check interactions before interpreting main effects in isolation, since a significant interaction can make a main effect's sign flip depending on the other factor's level (not the case here, since no interaction was significant).

Source. Banks, Carson, Nelson & Nicol, Discrete-Event System Simulation, ch. 12 (experimental design: $2^k$ factorial designs, ANOVA, screening experiments).

← Question 9 Back to the paper →

Question 10 of 10: 2 3 Full Factorial Screening Experiment for Process Throughput

Question 10 (Part C.2): 23 Full Factorial Screening Experiment for Process Throughput (20 marks)

Question 10 (Part C.2): 2³ Full Factorial Screening Experiment for Process Throughput (20 marks)