Regularization Methods for Nonlinear Least Squares Problems. Part I: Exactly Rank-deficiency

An optimization problem that does not have an unique local minimum is often very difficult to solve. For a nonlinear least squares problem this is the case when the Jacobian is rank deficient in a neighborhood of a local minimum. Moreover, a Gauss-Newton method such as Levenberg-Marquardt will have very slow convergence for such a problem. We analyze these problems where the Jacobian is rank deficient and suggest other problem formulations more suitable for Gauss-Newton methods. The two methods we propose are a truncated Gauss-Newton method and a Gauss-Newton method based on the Tikhonov regularized nonlinear least squares problem. We test the methods on artificial problems where the rank of the Jacobian and the nonlinearity of the problem may be chosen making it possible to show the different features of the problem and the methods. The conclusion from the analysis and the tests is that the two methods have similar local convergence properties. The method based on Tikhononv regulari..