Sequential Estimation of Variance in Steady-State Simulation

TR-COSC 03/08Today, many studies of communication networks rely on simulation conducted to assess their performance. Steady-state simulation is used to draw conclusions about the long-run behaviour of stable systems. Current methodology of analysis of output data from steady-state simulation focuses almost exclusively on the offline estimation of the steady-state means of the parameters under investigation. Thus, the literature on “variance estimation” mostly deals with the estimation of the variance of the mean, which is needed to construct a confidence interval of the estimated mean values. So far, little work has been done on the estimation of the steady-state variance of simulated processes. In the performance analysis of communication networks, we find applications where the packet delay variation or jitter is of interest. In audio or video streaming applications, networking packets should take approximately the same time to arrive at their destination; the delay itself is less important (see e.g. Tanenbaum, 2003). To find the jitter of a communication link, the variance of the packet delay times needs to be estimated. The theoretical background of this research includes sequential steady-state simulation, stochastic processes, basic results on the estimation of the steady-state mean, and stochastic properties of the variance. These are briefly summarised in Chapter 2. The aim of this research is the sequential (online) estimation of the steady-state variance, along with the variance of the variance which is used to construct a confidence interval of the estimate. To this end, we propose and evaluate several variance estimators in Chapter 4. As an additional focus, we investigate the initial transient period of simulation output, and try to find methods of automated, sequential detection of the end of this period in Chapter 3. The research that led to this report was based on Akaroa2, an automated, parallel simulation controller developed at the University of Canterbury in Christchurch, New Zealand. We implement the proposed variance estimators with the help of the Akaroa2 framework, and assess their performance experimentally. The results of these experiments are presented in Chapter 5