Regularization Class of Statistical Models

We begin with a few historical remarks about what might be called the regularization class of statistical model building methods, which include penalized likelihood, support vector machines, robust and quantile nonparametric regression, etc., etc, and the problem of tuning them, spending a little too much time on methods related to Generalized Cross Validation. After that we discuss an approach to variable and pattern selection given very large attribute vectors, based on the LASSO (that is, l1 penalties) that differs from most approaches to this problem in that it is a mostly global, rather than a sequential, or greedy algorithm, for finding patterns in the data that most influence an outcome