Least Squares and Maximum Likelihood

Contents 1 The Linear Least Squares Problem 3 1.1.1 Under and overspeci ed models . . . . . . . . . . . . . . . 7 1.1.2 Stepwise regression . . . . . . . . . . . . . . . . . . . . . . 9 1.1.3 The lasso . . . . . . . . . . . . . . . . . . . . . . . . . . . 10 1.1.4 The case of an intercept term . . . . . . . . . . . . . . . . 12 1.2 The generalised least squares problem . . . . . . . . . . . . . . . 13 1.3 Minimum variance estimates . . . . . . . . . . . . . . . . . . . . 15 1.3.1 Gauss-Markov theory . . . . . . . . . . . . . . . . . . . . 15 1.3.2 Prediction of random eects . . . . . . . . . . . . . . . . . 17 1.3.3 Mixed models . . . . . . . . . . . . . . . . . . . . . . . . . 21 1.3.4 The projection theorem . . . . . . . . . . . . . . . . . . . 23 1.4 Estimation of dynamic models . . . . . . . . . . . . . . . . . . . 25 1.4.1 The ltering problem . . . . . . . . . . . . . . . . . . . . 27 1.4.2 The smoothing problem . . . . . . . . . . . . . . . . . . . 29 1.5 Mean models . . . .