HIGH DIMENSIONAL REGRESSION USING THE SPARSE MATRIX TRANSFORM (SMT)

Regression from high dimensional observation vectors is par-ticularly difficult when training data is limited. More specifi-cally, if the number of sample vectors n is less than dimension of the sample vectors p, then accurate regression is difficult to perform without prior knowledge of the data covariance. In this paper, we propose a novel approach to high dimen-sional regression for application when n < p. The approach works by first decorrelating the high dimensional observation vector using the sparse matrix transform (SMT) estimate of the data covariance. Then the decorrelated observations are used in a regularized regression procedure such as Lasso or shrinkage. Numerical results demonstrate that the proposed regression approach can significantly improve the prediction accuracy, especially when n is small and the signal to be pre-dicted lies in the subspace of the observations corresponding to the small eigenvalues. Index Terms — High dimensional regression, covariance estimation, sparse matrix transfor