Robust nonparametric regression by controlling sparsity

Nonparametric methods are widely applicable to statistical learn-ing problems, since they rely on a few modeling assumptions. In this context, the fresh look advocated here permeates benefits from variable selection and compressive sampling, to robustify non-parametric regression against outliers. A variational counterpart to least-trimmed squares regression is shown closely related to an 0-(pseudo)norm-regularized estimator, that encourages sparsity in a vector explicitly modeling the outliers. This connection suggests efficient (approximate) solvers based on convex relaxation, which lead naturally to a variational M-type estimator equivalent to Lasso. Outliers are identified by judiciously tuning regularization parame-ters, which amounts to controlling the sparsity of the outlier vector along the whole robustification path of Lasso solutions. An im-proved estimator with reduced bias is obtained after replacing the 0-(pseudo)norm with a nonconvex surrogate, as corroborated via simulated tests on robust thin-plate smoothing splines. Index Terms — Robustness, nonparametric regression, outlier rejection, sparsity, Lasso