On universal oracle inequalities related to high dimensional linear models

This paper deals with recovering an unknown vector θ from the noisy data Y = Aθ + σξ, where A is a known m × n- matrix and ξ is a white Gaussian noise. It is assumed that n is large and A may be severely ill-posed. Therefore, in order to estimate θ, a spectral regular-ization method is used, and our goal is to choose its regularization pa-rameter with the help of the data Y. For spectral regularization meth-ods related to the so-called ordered smoothers [see [Kneip (1994)]] we propose new penalties in the principle of empirical risk minimization. The heuristical idea behind these penalties is related to balancing ex-cess risks. Based on this approach, we derive a sharp oracle inequality controlling the mean square risks of data-driven spectral regularization methods. 1 Introduction and main results In this paper, we consider a classical problem of recovering an unknow