On estimating the score function and the choice of the smoothing parameter with applications to adaptive estimations

Estimation of the derivative of the log density, or score, function is central to much of recent work on adaptive estimation of econometric models. Most existing score function estimation methods approach the problem by differentiating the logarithm of an estimated density function, such as the kernel estimate. Cox (1985) proposed a direct method of estimating score functions using smoothing spline techniques. Under mild regularity conditions, Cox's estimate is consistent and achieves the optimal rate of convergence. This approach is appealing not only because it is based directly on penalized likelihood methods for the score function rather than some other related quantity.As in any smoothing problem there is a choice on smoothing parameter, which determines the trade-off between data-fidelity and smoothness of fit. Conventional data-driven choice criteria using cross validation or generalized cross-validation turn out to be extremely inefficient and expensive in the spline smoothing context due to the fact that the cubic spline coefficients of all but one observations have to be re-evaluated for each data point. An alternative, the pseudo maximum likelihood estimator, which maximizes the pseudo likelihood function using a recovered density function is introduced and shown to perform well in Monte-Carlo simulations.A small scale Monte-Carlo simulation is carried out to evaluate the relative performances of the score function estimation using smoothing spline, conventional kernel and a bi-kernel variation which uses different smoothing parameters for the numerator and denominator.Smoothing spline score estimation as well as the conventional kernel score estimation is then applied to adaptive M-estimation (Bickel(1982) and Manski(1984)) and adaptive L-estimation (Portnoy and Koenker(1988)) of the parameters of the linear regression model to study the performances of these adaptive estimates in small sample situations. Such adaptive estimates of the linear model are also compared with the estimates obtained by semi-nonparametric methods of Gallant and Nychka(1987) based on Hermite polynomials. It is well known that density estimate using Hermite series expansion is not consistent for distributions that cannot be determined by their moments, like the Cauchy distribution. This can lead to very poor estimate of the model parameters particularly if the error distribution has heavy tails, or if there is a strong asymmetry in the error distribution. Monte-Carlo results confirm that semi-nonparametric estimation based on Hermite polynomials calls for a critical reappraisal.U of I OnlyETDs are only available to UIUC Users without author permissio