Variable selection in semi-parametric models

Graduation date: 2014We consider two semiparametric regression models for data analysis, the stochastic additive model (SAM) for nonlinear time series data and the additive coefficient model (ACM) for randomly sampled data with nonparametric structure. We employ the SCAD-penalized polynomial spline estimation method for estimation and simultaneous variable selection in both models. It approximates the nonparametric functions by polynomial splines, and minimizes the sum of squared errors subject to an additive penalty on norms of spline functions. A coordinate-wise algorithm is developed for finding the solution for the penalized polynomial spline problem. For SAM, we establish that, under geometrically α-mixing, the resulting estimator enjoys the optimal rate of convergence for estimating the nonparametric functions. It also selects the correct model with probability approaching to one as the sample size increases. For ACM, we investigate the asymptotic properties of the global solution of the non-convex objective function. We establish explicitly that the oracle estimator is the global solution with probability approaching to one. Therefore, the global solution enjoys both model estimation and selection consistency. In the literature, the asymptotic properties of local solutions rather than global solutions are well established for non-convex penalty functions. Our theoretical results broaden the traditional understandings of the penalized polynomial spline method. For both models, extensive Monte Carlo studies have been conducted and show the proposed procedure works effectively even with moderate sample size. We also illustrate the use of the proposed methods by analyzing the US unemployment time series under SAM, and the Tucson housing price data under ACM