Linear restrictions, rank reduction, and biased estimation in linear regression

AbstractFour types of biased estimators of thek × 1 coefficient vector in the linear regression model are considered: (1) the minimum-variance conditionally unbiased affine estimator subject tor < k independent linear restrictions; (2) the “blown-up” aggregative estimator obtained by partitioning the independent variables into / groups, replacing thek independent variables byl linear combinations of them, and “blowing up” the resultingl × 1 estimator into ak × 1 estimator of the original coefficient vector (a common procedure in econometrics); (3) a generalization of the Marquardt procedure of replacing then × k observation matrix by its best approximation (in terms of the Frobenius norm) by ann × k matrix of reduced rankl, and taking the generalized inverse of this matrix; (4) a generalization of the ridge estimator involving an approximative linear restriction. It is shown that procedures (1)–(3) are formally equivalent; e.g., procedure (2) may be considered either as equivalent to estimation subject to a set ofr = k − l linear restrictions on the coefficient vector, or as equivalent to projecting then × k observation matrix to ann × k matrix of reduced rank,l, and taking its generalized inverse. A comparison of these three procedures is made with procedure (4); some sufficient conditions are obtained under which the matrix mean-square error of estimator (1) is lower than the corresponding generalized ridge estimator obtained by procedure (4)