Sparse Single-Index Model

Let (X,Y) be a random pair taking values in Rp×R. In the so-called single-index model, one has Y = f ⋆(θ⋆TX)+W, where f ⋆ is an unknown univariate measurable function, θ ⋆ is an unknown vec-tor in Rd, andW denotes a random noise satisfying E[W |X] = 0. The single-index model is known to offer a flexible way to model a variety of high-dimensional real-world phenomena. However, de-spite its relative simplicity, this dimension reduction scheme is faced with severe complications as soon as the underlying dimension becomes larger than the number of observations (“p larger than n ” paradigm). To circumvent this difficulty, we consider the single-index model estimation prob-lem from a sparsity perspective using a PAC-Bayesian approach. On the theoretical side, we offer a sharp oracle inequality, which is more powerful than the best known oracle inequalities for other common procedures of single-index recovery. The proposed method is implemented by means of the reversible jump Markov chain Monte Carlo technique and its performance is compared with that of standard procedures