Transdimensional sampling algorithms for Bayesian variable selection in classification problems with many more variables than observations

One flexible technique for model search in probit regression is Markov chain Monte Carlo methodology that simultaneously explores the model and parameter space. The reversible jump sampler is designed to achieve this simultaneous exploration. Standard samplers, such as those based on MC3, often have low model acceptance probabilities when there are many more regressors than observations. Simple changes to the form of the proposal leads to much higher acceptance rates. However, high acceptance rates are often associated with poor mixing of chains. This suggests defining a more general model proposal that allows us to propose models "further" from our current model. We design such a proposal which can be tuned to achieve a suitable acceptance rate for good mixing (rather like the tuning of a random walk proposal in fixed dimension problems). The effectiveness of this proposal is linked to the form of the marginalisation scheme when updating the model and we propose a new efficient implementation of the automatic generic transdimensional algorithm of Green (2003), which uses our preferred marginalisation. The efficiency of these methods is compared with several previously proposed samplers on some gene expression data sets. The samplers considered are: the data augmentation method of Holmes and Held (2006), the automatic generic transdimensional algorithm of Green (2003) and the efficient jump proposal methods of Brooks et al (2003). Finally, the results of these applications lead us to propose guidelines for choosing between samplers