Understanding Unexpected Behaviors in Exploratory Simulations

Simulations and computational models have become the common tool of subject matter experts (SMEs) in a variety of disciplines to explore systems with inherent uncertainty. Predictions from these models and simulations have entered the mainstream of critical public policy and research decision-making practices. Unfortunately, methods for gaining insight into unexpected simulation outcomes have not kept pace. SMEs need to understand and explain unexpected behaviors in exploratory simulations to determine if the behaviors reflect an error or if they represent new knowledge in the discipline. Common practice is to apply classic debugging techniques to identify the program statements and interactions that lead to the unexpected behaviors. This practice is largely manual, it can consume years of effort, and it will not scale as models increase in complexity. Automation of at least a portion of the process has become essential. The automated process proposed here, Bayesian Program Slicing (BPS), will combine program slicing and Bayesian networks in a novel manner to identify program statements that are relevant to understanding unexpected behaviors. BPS will facilitate focusing SME attention on understanding and explaining the interactions of program statements whose execution results in variable state changes that are most relevant to the unexpected behaviors. Issues that make the proposed approach research challenging include: identifying prior knowledg