The Theory and Application of Penalized Least Squares Methods or Reproducing Kernel Hilbert Spaces Made Easy

The popular cubic smoothing spline estimate of a regression function is the minimizer of X j d j (Y j \Gamma ¯(t j )) 2 + Z b a h ¯ 00 (t) i 2 dt; where (Y j ; t j ) are the data and the d j 's are positive weights. However, sometimes the data are related to the function of interest ¯ in another way, i.e., E(Y i ) = F i (¯) for some known F i 's. And sometimes, one may wish to replace R (¯ 00 ) 2 with another expression. This paper discusses the solution for these generalizations, that is, the minimization of X j d j (Y j \Gamma L j (¯)) 2 + Z b a h (L¯)(t) i 2 dt: Here, L is a linear differential operator of order m 1: (L¯)(t) = ¯ (m) (t) + P m\Gamma1 j=0 w j (t)¯ (j) (t). This paper outlines basic theory for this general minimization problem, and provides explicit directions for calculating the minimizer. The minimizer depends on the easily calculated reproducing kernel associated with L. 2 Introduction The cubic smoothing spline, a popular regre..