Adviser Graduate Program in Statistics c © Copyright by

We consider statistical procedures for feature selection defined by a family of regu-larization problems with convex piecewise linear loss functions and penalties of l1 or l∞ nature. For example, quantile regression and support vector machines with l1 norm penalty fall into the category. Computationally, the regularization problems are linear program-ming (LP) problems indexed by a single parameter, which are known as ‘parametric cost LP ’ or ‘parametric right-hand-side LP ’ in the optimization theory. Their solution paths can be generated with certain simplex algorithms. This work exploits the connection between the family of regularization methods and the parametric LP theory and lays out a general simplex algorithm and its variant for generating regularized solution paths for the feature selection problems. The significance of such algorithms is that they allow a complete ex-ploration of the model space along the paths and provide a broad view of persistent features in the data. The implications of the general path-finding algorithms are outlined for various statistical procedures, and they are illustrated with numerical examples. ii This is dedicated to my parents, my wife Youlan Rao, and my daughter Rebecca R. Yao. iii ACKNOWLEDGMENTS I would like to express my appreciation to my adviser, Dr. Yoonkyung Lee, for he