23-Ind-A6 Systems Simulation · Undated paper

Question 6 of 10: Run Length — Warm-Up Determination for a Volatile Series

Nivaar worked solution (AI-drafted; not reviewed by a licensed engineer)

Question 6 (Part B.1): Run Length — Warm-Up Determination for a Volatile Series (15 marks)

Question text not reproduced: the examination questions are © Engineers and Geoscientists BC. Open the official past paper (linked at the top of this page) to read the question, then follow the worked solution below.

Given. A single simulated output series, patient wait (in days) at a hospital emergency department, plotted over $30$ simulated months (reproduced below).

[Figure not reproduced: Simulated patient wait (days) vs. simulated month, redrawn from the source figure: a persistent, noisy upward trend with no visible plateau across the 20 months actually plotted. See the official exam paper.]

Find. (a) why run length matters; (b) run-length considerations under batch means vs. replication/deletion; (c) an algorithm to check both sets of run assumptions; (d) why the warm-up period is hard to read off this particular graph; (e) whether a 12-month replication/deletion run length is a good compromise.

Approach. Answer (a)–(c) and (e) conceptually from simulation output-analysis theory; ground (d) directly in the shape of the plotted series.

(a) Why run length matters. A simulation started from arbitrary initial conditions (e.g. an empty emergency department) passes through a transient phase before its output distribution settles into steady state. A run that is too short is dominated by that transient and by sampling noise, so any performance estimate (mean wait, utilization, …) built from it can be badly biased and imprecise — long enough runs are what let the estimator converge on the system's true long-run behaviour rather than an artifact of how the run happened to start and how long it happened to go.

(b) Run length under each method. Under batch means, one very long run is made, the warm-up is deleted once at the start, and the remaining output is split into contiguous batches whose means are treated as approximately independent; the run length must be long enough both to reach steady state once and to contain enough batches, each long enough to be nearly uncorrelated with its neighbours. Under replication/deletion, several independent shorter runs are made (each with its own random-number stream), and the warm-up is deleted separately from every replication; here the run length of each individual replication must, on its own, be long enough to both pass through the warm-up and accumulate a representative post-warm-up sample — too short a per-replication length cannot be rescued simply by adding more replications, since every replication repeats the same inadequate post-warm-up window.

(c) An algorithm covering both methods. (i) Run several independent pilot replications and average them at each time index to get a cross-replication mean series $\bar Y(t)$, cancelling within-run noise while preserving any common transient trend; (ii) apply a graphical smoother (e.g. Welch's moving average) to $\bar Y(t)$ and identify the point $l$ beyond which the smoothed curve stops trending — this $l$ is the warm-up length, applicable to both methods; (iii) for replication/deletion, delete the first $l$ observations of every replication and build a $t$-based CI on the remaining, already-independent replicate means, checking the CI's relative half-width against a target precision and adding replications if it is too wide; (iv) for batch means, delete $l$ once from the single long run, split the remainder into batches, and check the batches are approximately uncorrelated (e.g. a lag-1 autocorrelation test on the batch means) before applying the same $t$-based CI — if batches are still correlated, lengthen the batch size (equivalently, the total run length) rather than trusting the interval.

(d) Why the warm-up is hard to read off this graph. Welch's method (and any graphical warm-up read) relies on the smoothed series levelling off into a roughly flat band once steady state is reached. This series never does that within the $20$ months actually plotted: after an early low, mildly noisy stretch it climbs almost continuously, punctuated by sharp local peaks and drops (e.g. the run-up to $\sim$month 15, the dip afterward, and the second peak near month 19) that look just as plausibly like ordinary steady-state noise as like a still-ongoing transient. With no visible plateau to anchor on, any chosen cutoff is essentially a guess between two competing, equally defensible readings: either the whole visible window is still transient (the system is genuinely non-stationary, e.g. a steadily growing patient population), or steady state has a large enough natural variance that the apparent trend is just noise — and the graph alone cannot distinguish these two explanations.

(e) Is a 12-month deletion/replication run a good compromise? No. From the plotted series, month 12 falls well before the region where the curve shows any hint of levelling — the sharpest rises and the two highest peaks all occur after month 12 (months 13–19). Truncating every replication's warm-up-plus-observation window at 12 months risks cutting the run off while the process is still in (or approaching) its most volatile, possibly still-transient phase, so the reported average would understate both the mean wait and its true variability. $$\boxed{\text{No} \;-\; 12\ \text{months is not clearly long enough to have reached steady state, given the graph}}$$ A defensible substitute is to extend the pilot runs well past month 20, look for a genuine plateau in the smoothed cross-replication series before committing to any fixed run length, and treat the possibility that the system has no true steady state (ongoing demand growth) as a real alternative that would need a different analysis (e.g. explicitly time-varying performance measures) rather than a single warm-up cutoff.