ANALYSIS OF DISTRIBUTIONS IN FACTORIAL EXPERIMENTS

The output from simulation factorial experiments can be complex and may not be amenable to standard methods of estimation like ANOVA. Two particular difficulties are: (i) the simulation output may not satisfy normality assumptions, and (ii) there may be differences in output at different factor combinations, but these are not simply differences in means. For the situation where there are replicated observations we show that the Cramer-von Mises goodness of fit statistic can be generalised to handle both these difficulties, yielding a similar but potentially more sensitive analysis to that offered by ANOVA. Moreover if the method is applied to ranked data rather than original observations then the method becomes distribution free. For this case we give the asymptotic theory; however its advantage is that for small sample sizes, Monte-Carlo sampling can be used to directly generate arbitrarily accurate critical test null values in online analysis. The method is illustrated with an example based on a real man in the loop simulation trial where operators carried out self assessment of the workload that they experienced under different operating conditions