1A model selection approach for local learning

Local learning techniques, for each query, extract a predic-tion interpolating locally the neighboring examples which are considered relevant according to a distance measure. As other learning approaches, the local learning procedure can be conveniently decomposed into a parametric identification and a structural identification. While parametric identifi-cation is reduced to a linear regression, structural identifi-cation requires that the designer perform a certain number of choices. In this paper we focus on an automatic query-by-query selection of the bandwidth, a structural parameter which plays a major role in the final performance. We pro-pose a local method where, for each query, different model candidates are first generated, then assessed and finally se-lected. We introduce in the context of local learning the re-cursive least squares algorithm as an efficient way to gen-erate local models. Moreover, local cross-validation is used as an economic way to validate different alternatives. As far as model selection is concerned, the winner-takes-all strat-egy and a local combination of the most promising models are explored. The method proposed is tested on six different datasets and compared with state-of-the-art approaches. 1