Partial Least Squares vs. Lanczos Bidiagonalization I: Analysis of a Projection Method for Multiple Regression

Multiple linear regression is considered and the partial least squares method (PLS) for computing a projection onto a lower-dimensional subspace is analyzed. In the analysis we use the equivalence to Lanczos bidiagonalization. In particular we illustrate, using singular value analysis and Krylov subspaces, why, in many cases, PLS gives a faster reduction of the residual than standard principal components regression. Our analysis also shows why in some cases the dimension of the subspace, given by PLS, is not as small as desired