23-Ind-A6 Systems Simulation · May 2018

Question 6 of 9: Replication Length, Run Strategy

Nivaar worked solution (AI-drafted; not reviewed by a licensed engineer)

Notes on this paper

National Exams — May 2018 — 17-Ind-A6 Systems Simulation. Three-hour, closed-book exam; one of two permitted calculators (Sharp or Casio), one 8.5″×11.0″ aid sheet (both sides). Format: three sections — Section A (four concept questions, candidates choose any two, 10 marks each, 20 marks total), Section B (three methods questions built around one continuing warehouse-simulation case study, candidates choose any two, 15 marks each, 30 marks total), Section C (two applications questions, candidates choose any one, 20 marks each). All nine questions are solved below for completeness. Two source anomalies are flagged where they occur: the front-page summary table states Section A is "Do 2 of 3," while Section A's own instructions and its four printed question sets read "two of the following four" — the printed four-question section is answered in full here; and Part C Question 2's sub-parts (a)–(c) are never printed anywhere in the paper, even though the results text for (d)–(f) explicitly refers back to "the factorial design matrix in (a)." Two-page Normal-distribution tables were supplied with the exam; the values below are the same table values obtained by direct computation.

Reference texts: Banks, Carson, Nelson & Nicol, Discrete-Event System Simulation (5th ed., Pearson) — random-number/random-variate generation, input modeling and goodness-of-fit testing, output analysis (warm-up, replication length, batch means vs. replication/deletion), and comparing alternative systems; Montgomery, Design and Analysis of Experiments (current ed., Wiley) — single-factor ANOVA, multiple comparisons, and 2^k factorial designs with interaction analysis (Part C).

Question 6 (Part B.2): Replication Length, Run Strategy (15 marks)

Question text not reproduced: the examination questions are © Engineers and Geoscientists BC. Open the official past paper (linked at the top of this page) to read the question, then follow the worked solution below.

Given. Ten pilot replications of 30 simulated days each, total throughput per replication:

Pilot 30-day replication throughput
Rep	1	2	3	4	5	6	7	8	9	10
Throughput	3500	3851	4522	4756	4872	4020	4618	4968	4632	4548

Find. (a) proof that 30 days is an insufficient run length; (b) an algorithm for determining a proper run length; (c) batch means vs. replication/deletion, compared; (d) whether switching to replication/deletion at 30 days would solve (a).

Approach. Form a 95% CI on the pilot mean and judge its relative precision against a standard target; from there, prose covers the sequential-length algorithm, the strategy comparison, and the (d) conclusion.

(a) Test the pilot data's precision. With $n=10$: $\bar x=4428.7$, $s=478.17$ (a range of $3500$ to $4968$ — a $42\%$ spread across nominally identical 30-day runs). The 95% CI: $$\bar x \pm t_{0.025,9}\frac{s}{\sqrt n} = 4428.7 \pm 2.262\times\frac{478.17}{\sqrt{10}} = 4428.7\pm342.1.$$ Relative half-width $=342.1/4428.7=7.7\%$, comfortably above a common $5\%$ target-precision criterion for a production estimate. (Check: the question supplies no explicit target precision $\gamma$; $5\%$ is adopted here as the standard textbook benchmark.) The 30-day runs are producing an estimate that is far too imprecise to trust: $$\boxed{\text{30-day replication length is insufficient}}$$
(b) Algorithm for a proper run length. A sequential relative-precision procedure: (i) fix a target relative precision $\gamma$ (e.g. $5$–$10\%$) and confidence $1-\alpha$; (ii) run an initial $n_0$ replications (or an initial run length) past the already-identified warm-up; (iii) compute the running half-width $t_{\alpha/2,n-1}s/\sqrt n$ and its ratio to the running mean; (iv) if that ratio exceeds $\gamma$, either lengthen EACH run (if within-run non-stationarity is suspected, as it is here) or add more independent replications using $n^\ast\approx n_0\left(t_{\alpha/2,n_0-1}s/(\gamma\bar x)\right)^2$ — the same formula applied numerically in Question 3(a); (v) repeat until the criterion is met, re-checking that the underlying batches/replications remain approximately independent (e.g. a lag-1 autocorrelation check) before trusting the interval.

(c) Batch means vs. replication/deletion.

Run-strategy comparison
Strategy	Implementation	Advantage	Disadvantage
Batch means	one long run; delete warm-up once; split the remaining output into contiguous batches; treat batch averages as approximately independent	only one warm-up ever discarded — efficient use of simulated time	batches can remain autocorrelated unless long enough; batch size must be chosen carefully
Replication/deletion	several independent runs (separate random-number streams); delete warm-up in EACH run; average the post-warm-up portion of each	replicate means are independent by construction — ordinary $t$-based CI applies directly; trivially parallelizable	re-runs (and re-discards) the warm-up in every replication — wastes computation relative to batch means

(d) Would replication/deletion at 30 days fix (a)? No. Part (a)'s finding is about whether a SINGLE 30-day run is long enough to have settled into a trustworthy steady-state estimate — and the ten pilot values already ARE ten independent 30-day replications with their own warm-up (implicitly) removed, which is exactly what produced the wide CI in (a). Re-labelling the same 30-day-length runs as "replication/deletion" changes nothing about their individual length; replication/deletion only guarantees INDEPENDENCE between runs (already present here), not adequate LENGTH within each run. $\boxed{\text{No} - \text{the fix must be a longer run (or a re-derived, much larger replication count), not a change of run-management strategy.}}$

Item	Result
(a) 95% CI, relative precision	$4428.7\pm342.1$ (7.7%) — exceeds a 5% target — insufficient
(b) fix	sequential relative-precision procedure, extend length and/or replications
(d) does relication/deletion @ 30d solve (a)?	No — independence was never the problem; length is

Concept & Theory

Topic. Replication-length adequacy, sequential precision procedures, and batch-means vs. replication/deletion.

Key relations. $CI=\bar x\pm t_{\alpha/2,n-1}s/\sqrt n$; sequential $n^\ast\approx n_0\left(ts/(\gamma\bar x)\right)^2$.

Why this works. Precision of an output estimator depends both on how many (quasi-)independent observations are averaged AND on how representative each individual observation already is of steady state; a wide relative CI built from replications that are already independent points squarely at the second cause.

Common pitfall. Treating "more replications" as a universal fix — if each individual run hasn't reached a representative state, no number of additional SHORT replications removes that bias; only more replications of an already-adequate length reduces variance the way the formula assumes.

Source. Banks, Carson, Nelson & Nicol, Discrete-Event System Simulation, ch. 9 (batch means, replication/deletion, sequential procedures for a desired precision).

← Question 5 Next: Question 7 →