Goodness-of-Fit Testing Using Cross-Validation Bayes Factors

Statistical methods for selecting between two competing models have a long and storied history from both the frequentist and Bayesian perspectives. That being said, there are known limitations that exist when using frequentist tests based on P-values for model selection. Therefore, we prefer to take a Bayesian approach to model selection that utilizes Bayes factors. In this research, we consider two different model selection problems: multivariate nonparametric goodness-of-fit and comparing two parametric models. For both problems, we propose intuitive and computationally simple model selection methods that take advantage of data splitting and cross-validation Bayes factors. Bayesian multivariate nonparametric goodness-of-fit is a difficult problem. The alternative model often requires an infinite-dimensional prior distribution that makes computation of the marginal likelihood complex. By applying data splitting, we are able to form a nonparametric alternative model using the familiar multivariate kernel density estimate and compute a cross-validation Bayes factor very easily. As for comparing two parametric models (either nested or non-nested), difficulties can arise when formulating prior distributions or approximating marginal likelihoods for either model. We can avoid both of these concerns by computing a prior-free cross-validation Bayes factor by using data splitting. These Bayes factors depend solely on computing maximum likelihood estimates and evaluating likelihood functions. In both scenarios, we show that our cross-validation Bayes factors are consistent at an exponential rate, regardless of which hypothesis is true. This includes the traditionally difficult case where the smaller of two nested parametric models is true. We also provide numerous simulation studies and real data analyses to explore performance and practical application of these methods