Automatic Differentiation: Overview and Application to Systems of Parameterized Nonlinear Equations

This work was also published as a Rice University thesis/dissertation: http://hdl.handle.net/1911/13777Automatic Differentiation is a computational technique that allows the evaluation of derivatives of functions defined by computer programs. Derivatives are calculated by applying the chain rule of differential calculus to the sequence of elementary computations involved in the program. In this work, an overview of the theory and implementation of automatic differentiation is presented, as well as a description of the available software. An application of automatic differentiation in the context of solving systems of parameterized nonlinear equations is discussed. In this application, the "differentiated" functions are implementations of Newton's method and Broyden's method. The iterates generated by the algorithms are differentiated with respect to the parameters. The results show that whenever the sequence of iterates converges to a solution of the system, the corresponding sequence of derivatives (computed by automatic differentiation) also converges to the correct value. Additionally, we show that the "differentiated" algorithms can be successfully employed in the solution of parameter identification problems via the Black-Box method