Shrinkage structure in biased regression

Biased regression is an alternative to ordinary least squares (OLS) regression, espe-cially when explanatory variables are highly correlated. In this paper, we examine the geometrical structure of the shrinkage factors of biased estimators. We show that, in most cases, shrinkage factors cannot belong to [0, 1] in all directions. We also compare the shrinkage factors of ridge regression (RR), principal component regression (PCR) and partial least squares regression (PLSR) in the orthogonal di-rections obtained by the signal-to-noise ratio (SNR) algorithm. In these directions, we find that PLSR and RR behave well whereas shrinkage factors of PCR have an erratic behaviour. Key words: Biased regression, regression on components, shrinkage factors, signal-to-noise ratio, James-Stein estimator