Forward Stagewise Regression and the Monotone Lasso

We consider the least angle regression and forward stagewise algorithms for solving penalized least squares regression problems. In Efron et al. (2004) it is proven that the least angle regression algorithm, with a small modification, solves the lasso (L 1 constrained) regression problem. Here we give an analogous result for incremental forward stagewise regression, showing that it fits a monotone version of the lasso. We also study a condition under which the coe#cient paths of the lasso are monotone, and hence the di#erent algorithms all coincide. Finally, we compare the lasso and forward stagewise procedures in a simulation study involving a large number of correlated predictors