23-Ind-A6 Systems Simulation · December 2018

Question 4 of 8: Simulation Run Control — Replications, Warm-Up, Batch Means

Nivaar worked solution (AI-drafted; not reviewed by a licensed engineer)

Question 4 (Part B.1): Simulation Run Control — Replications, Warm-Up, Batch Means (20 marks)

Question text not reproduced: the examination questions are © Engineers and Geoscientists BC. Open the official past paper (linked at the top of this page) to read the question, then follow the worked solution below.

Given. Current setup: $n=7$ "replications" of 5 days (120 h) each, warm-up 2.5 h applied at the start of every one, Statistics reinitialized between reps but System state carried forward; current 95% CI on mean wait $=52\pm4.5$ min. Standard: mean wait $\le 45$ min. Target precision: half-width $\le 2$ min at 95% confidence (19/20).

Find. (a)–(e) as posed.

Approach. Back out the working standard deviation $s$ from the current CI ($h=t_{0.025,n-1}s/\sqrt n$), then use it to evaluate the proposed $n=5$ reduction and to solve iteratively for the $n$ that meets the $\pm2$-minute target.

1. Recover the working $s$ from the current 7-run CI. $$s = \frac{h\sqrt n}{t_{0.025,6}} = \frac{4.5\sqrt7}{2.447} = \boxed{4.87\ \text{min}}.$$

(a) Cutting 7 runs to 5 — is it a good idea? No. Both factors that set the half-width move in the wrong direction when $n$ falls: $\sqrt n$ shrinks (widening $h$ directly) and the $t$-critical value grows as $df$ falls (from $t_{0.025,6}=2.447$ to $t_{0.025,4}=2.776$), widening it further. Using the recovered $s=4.87$: $$h(5) = t_{0.025,4}\,\frac{s}{\sqrt5} = 2.776\times\frac{4.87}{\sqrt5} = \boxed{6.04\ \text{min}} \;>\; h(7)=4.50\ \text{min}.$$ Robert's reasoning is backwards: dropping to 5 runs produces a wider, less precise CI (52 $\pm$ 6.0 rather than 52 $\pm$ 4.5), which does not "ensure the upper end covers the expected mean" — it only makes the estimate less trustworthy in every direction. Reducing replications never improves precision; only increasing them does (part e).

(b) Has warm-up been handled properly? Not correctly, given how the run is actually configured. With "System" not reinitialized between reps, the transport-centre queue genuinely never restarts — the whole 7-rep run is really one continuous 35-day process. Yet "Statistics" is cleared, and the 2.5-hour warm-up is applied, at the start of each of the 7 segments. Only the very first warm-up discards a genuine transient (the system starting empty and idle); the other six discard 2.5 hours of perfectly good steady-state data each time, for no reason, since the system was already warmed up from the prior segment. That is $6\times2.5=15$ hours of usable data thrown away. Robert should either (i) truly restart the system between reps (see part c) and keep a warm-up on every rep, or (ii) run one continuous simulation with a single warm-up at the very start and no warm-up applied again mid-run.

(c) Batch means vs. replication/deletion. Replication/deletion is appropriate when independent replications are cheap to generate (fresh random-number streams, tolerable to re-pay the warm-up cost each time) — it gives a genuinely i.i.d. sample of replication means, so the standard $t$-based CI Robert is already using is valid outright. Batch means is appropriate for one long run where restarting the system repeatedly is wasteful or the system's steady state is slow to reach — but it requires the batches to be large enough that adjacent batch means are effectively uncorrelated, which must be checked (see Question 5c) before trusting a $t$-CI on the batches. Robert's own configuration (System not reinitialized) already signals batch means was the intended method, and here the warm-up transient is short relative to a 5-day batch ($2.5$ h out of $120$ h, about 2%), so re-paying it per replication is not expensive either way. Recommendation: given the setup already carries system state forward, complete the batch-means implementation properly — a single continuous run, one warm-up, statistics reset once per 5-day batch thereafter, with an explicit lag-1 autocorrelation check on the resulting batch means before the $t$-CI is trusted (batch means is the more defensible reading of what Robert has actually configured; replication/deletion would instead require also reinitializing the System box, discarding the continuity his own run already assumes).

(d) Advice on run length. Run length must be long enough that (i) the chosen warm-up is a small, defensible fraction of each batch/replication (a pilot run with Welch's graphical procedure — plotting a moving average of the wait-time series and finding where it visibly flattens — is the rigorous way to size the warm-up rather than guessing 2.5 h), and (ii) each batch is long enough that batch-to-batch correlation is negligible: compute the lag-1 autocorrelation of successive batch means and, if it exceeds roughly $\pm1.96/\sqrt{n}$, double the batch length and re-batch, repeating until it is not significant. Only once both checks pass should the standard $t$-interval be trusted.

(e) Replications for $\pm2$-minute precision at 95%. Holding $s=4.87$ fixed (from the pilot 7 runs) and solving $t_{0.025,R-1}\,s/\sqrt R \le 2$ iteratively (both sides depend on $R$, so no closed form):

2. Decision. $R=25$ still gives $2.01>2$; $R=26$ is the first replication count with half-width $\le2$: $$R = \boxed{26\ \text{total replications}}\ (19\ \text{more than the current pilot of }7).$$