A Monte Carlo evaluation of different ways to estimate effect sizes in quasi -experiments and observational studies

This dissertation research is to understand the statistical biases in estimating parameters in linear statistical models for analyzing data from observational and quasi-experimental designs directed toward estimating the effect of interventions. Some of the parameters in these models can be interpreted as measures of the intervention effect. The statistical biases affect inference one may draw about effect size, for instance. Three variable types were simulated, Type [special characters omitted] variables were related to both outcome and treatment assignment, Type [special characters omitted] variables were related to outcome but not treatment, while Type [special characters omitted] variables were related to treatment but not outcome. This research examined ordinary least squares (OLS), pairwise propensity score matching (PSM), naive pairwise matching (NPM), naive triplet matching (NTM), Mahalanobis distance triplet matching, Horvitz-Thompson estimator (with and without common support enforced), and cluster analysis using Ward\u27s method (5, 7, and 9 cluster solutions). The author undertook a comprehensive simulation to determine which estimators yielded smallest biases and under what conditions. The simulations manipulated seven factors: sample size, number of variables in the true outcome equation, the sample size in intervention groups, degree of overlap in the distributions of covariates. The fifth factor manipulated was the kind of estimator used. The sixth and seventh conditions are predicated on a true outcomes model with variables of type [special characters omitted] and [special characters omitted]. In the seventh condition variables of type [special characters omitted] and [special characters omitted] are left in or out of the fitted model. In the eighth condition, the importance of variables [special characters omitted] are manipulated at three levels in the true outcome model. The overall analysis of bias suggests that OLS and PSM have no bias under these conditions with large samples. Results of this dissertation are to be expected given the model and the mathematics underlying the estimates. The most severe biases are associated with Mahalanobis Distance Triplet Matching and the Horvitz-Thompson estimator with the inverse propensity score serving as the weighting unit