An Aggregation Method for Sparse Logistic Regression

We demonstrate and analyze an aggregation method for sparse logistic regression in high-dimensional settings. This approach linearly combines the estimators from various logistic models with different sparsity patterns and can balance the predictive ability and model interpretability. We also study the Kullback-Leibler risk of the aggregation estimator and show that it is comparable to the risk of the best estimator based on a single logistic regression, chosen by an oracle. Numerical performance of the estimator is also investigated using both simulated and real data.