A BIVARIATE ANALYSIS OF SIMULATION OUTPUT FROM STOCHASTIC STRUCTURED NETWORKS

Techniques which use stochastic structured networks in project network simulation (e.g., GERT and VERT) have focused predominately on the modeling phase of the simulation effort. Relatively little attention has been given to procedures for the analysis of output or the validation of model. This study develops a formal approach to bivariate output analysis and validation for project network models. A four stage validation framework is proposed with the purpose of guiding modelers (users) along a process which increases their confidence in what the model predicts about the real system. These stages include: (1) content validity, (2) operational validity, (3) output verification, and (4) model effectiveness. Content validity assures the integrity of the model base and database. Operational validity preserves content validity over time by means of updating and review. Output verification and model effectiveness focus on the correct statistical interpretation of output. Output verification determines: (1) the number of simulation runs on the network to control the accuracy of the output, and (2) the correct form of probability statements on which inferences are based, shown to depend on the number of realizations expected of the real system. Model effectiveness determines the extent that the behavior of the model as a stochastic system supports predictions which are of interest to the modeler. Two statistical methodologies for measuring effectiveness on completion time and project outcome are developed. The first measures effectiveness in absolute terms using probability measures; the second measures effectiveness in relative terms using descriptive statistics based on the bivariate entropy function. The significance of this study follows from the potential consequences of making erroneous conclusions about the real system, whether caused by an invalid model or the misinterpretation of output from a valid model. These consequences include both the real cost of initiating or continuing a project believed in error to be an acceptable risk, or the opportunity cost of rejecting or discontinuing a project believed in error to be an unacceptable risk