On bandwidth selection problems in nonparametric trend estimation under martingale difference errors

International audienceIn this paper, we are interested in the problem of smoothing parameter selection in nonparametric curve estimation under dependent errors. We focus on kernel estimation and the case when the errors form a general stationary sequence of martingale difference random variables where neither linearity assumption nor ``all moments are finite" are required.We compare the behaviors of the smoothing bandwidths obtained by minimizing either the unknown average squared error, the theoretical mean average squared error, a Mallows-type criterion adapted to the dependent case and the family of criteria known as generalized cross validation (GCV) extensions of the Mallows' criterion. We prove that these three minimizers and those based on the GCV family are first-order equivalent in probability. We give also a normal asymptotic behavior of the gap between the minimizer of the average square error and that of the Mallows-type criterion. This is extended to the GCV family.Finally, we apply our theoretical results to a specific case of martingale difference sequence, namely the Auto-Regressive Conditional Heteroscedastic (ARCH(1)) process.A Monte-carlo simulation study, for this regression model with ARCH(1) process, is conducted