Thresholding-based Iterative Selection Procedures for Model Selection and Shrinkage

High-dimensional correlated data pose challenges in model selec-tion and predictive learning. The predictors can be naturally grouped in some applications where pursuing the between-group sparsity is preferred. Moreover, the problems of interest often go beyond Gaus-sian models. This paper provides a framework to tackle these chal-lenges. We derive an iterative thresholding technique which solves the penalized generalized linear model problem in general. We es-tablish rigorous convergence conditions that are much more relaxed than those given in the literature and practically result in a decrease of the number of iterations. Our theories allow for nonconvex penal-ties (including the l0-penalty) and arbitrarily grouped predictors. A nonconvex hard-ridge penalty is advocated for joint model selection and prediction. A novel selective cross-validation (SCV) scheme is proposed for parameter tuning. We study super-resolution spectrum estimation and analyze real microarray data to illustrate the proposed methodology.