The Risk of James-Stein and Lasso Shrinkage

This paper compares the mean-squared error (or c2 risk) of Ordinary Least-Squares, James-Stein, and Lasso shrinkage estimators in simple linear regression where the number of regressors is smaller than the sample size. We compare and contrast the known risk bounds for these estimators, which shows that neither James-Stein nor Lasso uniformly dominates the other. We investigate the finite sample risk using a simple simulation experiment. We find that the risk of Lasso estimation is particularly sensitive to coefficient parameterization, and for a significant portion of the parameter space Lasso has higher mean-squared error than OLS. This investigation suggests that there are potential pitfalls arising with Lasso estimation, and simulation studies need to be more attentive to careful exploration of the parameter space. ∗Research supported by the National Science Foundation. I gratefully thank Jen-Che Liao for excellent research assistance