Bayesian Ensemble of Regression Trees for Multinomial Probit and Quantile Regression

This dissertation proposes multinomial probit Bayesian additive regression trees (MPBART), ordered multiclass Bayesian additive classification trees (O-MBACT) and Bayesian quantile additive regression trees (BayesQArt) as extensions of BART - Bayesian additive regression trees for tackling multinomial choice, multiclass classification, ordinal regression and quantile regression problems. The proposed models exhibit very good predictive performances. In particular, ranking among the top performing procedures when non-linear relationships exist between the response and the predictors. The proposed procedures can readily be applied on data sets with the number of predictors larger than the number of observations. MPBART is sufficiently flexible to allow inclusion of predictors that describe the observed units as well as the available choice alternatives and it can also be used as a general multiclass classification procedure. Through two simulation studies and four real data examples, we show that MPBART exhibits very good out-ofsample predictive performance in comparison to other discrete choice and multiclass classification methods. To implement MPBART, the R package mpbart is freely available from CRAN repositories. When ordered gradation is exhibited by a multinomial response, ordinal regression is an appealing framework. Ensemble of trees models, while widely used for binary classification, multiclass classification and continuous response regression, have not been extensively applied to solve ordinal regression problems. This work fills this void with Bayesian sum of regression trees. The predictive performance of our ordered Bayesian ensemble of trees model is illustrated through simulation studies and real data applications. Ensemble of regression trees have become popular statistical tools for the estimation of conditional mean given a set of predictors. However, quantile regression trees and their ensembles have not yet garnered much attention despite the increasing popularity of the linear quantile regression model. This work proposes a Bayesian quantile additive regression trees model that shows very good predictive performance illustrated using simulation studies and real data applications. Further extension to tackle binary classification problems is also considered