On Consistency of Bayesian Variable Selection for High Dimensional Binary Regression and Classification

Modern data mining and bioinformatics have presented an impor-tant playground for statistical learning techniques, where the number of input variables is possibly much larger than the sample size of the training data. In supervised learning, logistic regression or probit re-gression can be used to model a binary output and form perceptron classification rules based on Bayesian inference. We use a prior to select a limited number of candidate variables to enter the model, ap-plying a popular method with selection indicators. We show that this 1 approach can induce posterior estimates of the regression functions that are consistently estimating the truth, if the true regression model is sparse in the sense that the aggregated size of the regression co-efficients are bounded. The estimated regression functions therefore can also produce ‘consistent ’ classifiers that are asymptotically opti-mal for predicting future binary outputs. These provide theoretical justifications for some recent empirical successes in microarray data analysis.