Efficient factor screening in simulation experiments

Factor screening is performed in simulation experiments to eliminate obviously unimportant factors/effects such that more detailed investigations can focus on the most influential ones. Screening experiments also help to reduce a complex system to a simpler one so the analysis becomes more transparent. Controlled Sequential Bifurcation (CSB) is a recently proposed screening method for discrete-event simulations. It is the first method to provide simultaneous Type I Error and power control of the classification results. In this research, Controlled Sequential Factorial Design (CSFD) is proposed to overcome the limitations of CSB. CSFD combines sequential hypothesis testing procedures with a traditional factorial design to control the Type I Error and power for each effect under heterogeneous variances conditions. Numerical study shows that CSFD can be more efficient than CSB in many scenarios and is generally more effective. There also exists a complementary relationship between CSB and CSFD, which further inspires us to explore a hybrid approach that integrates prescreening, CSB and CSFD procedures into one screening method. The hybrid approach provides a general framework to perform factor screening efficiently and effectively. The underlying idea is that after the prescreening stage, CSFD is only used to screen potentially important effects (typically a small percentage of all effects) and CSB only screens potentially unimportant effects. The existence of the prescreening stage not only eliminates the requirement of prior knowledge of the simulation system but also allows the subsequent screening procedures to perform in their optimal conditions. Therefore, the hybrid method has better overall efficiency than its component screening methods in general circumstances. Moreover, the hybrid method requires minimal assumptions and demonstrates robust performance with different system conditions, thus having more general applications. The challenge is to coordinate different procedures and provide the overall error control of the screening results