The Effect of Line-Search Parameters on the Numerical Performance of Multi-Step Quasi-Newton Methods

Over the past twelve years, multi-step quasi-Newton methods for the unconstrained optimization of a nonlinear function f (with gradient denoted by g) have been developed to the point where they exhibit substantial improvements in numerical performance when compared with the “industry standard ” BFGS method- see [1-4], for example. These methods are based on the use of simple interpolatory curves in the variable space. Until recently, the multi-step methods had always been implemented under the same line-search conditions as those commonly recommended for the BFGS method (here, si = xi+1 – xi is the step between the two most recent iterates): f(xi+1) ≤ f(xi) + αsiTg(xi), (1) siTg(xi+1) ≥ βsiTg(xi), (2) where α = 10-4 and β = 0.9. However, recent experiments with different values for the parameters α and β have demonstrated that the performance of some multi-step methods can achieve further, significant gains by means of judicious selection of these parameters (more so than is possible with the BFGS method). In particular, these methods are able to tolerate (an