Setting up alternating least squares and iterative majorization algorithms for solving various matrix optimization problems

A general procedure is described for setting up monotonically convergent algorithms to solve some general matrix optimization problems, if desired, subject to a wide variety of constraints. An overview is given of a number of ready-made building blocks (derived in earlier publications) from which concrete algorithms are set-up with little effort. These algorithms are based on alternating least squares (block relaxation) and iterative majorization. It is demonstrated how the construction of an algorithm for a particular problem that falls in one of the classes of optimization problems under study, reduces to a simple combination of tools. Also, a procedure is reviewed for setting up a weighted least squares algorithm for any problem for which an unweighted least squares solution is available. All procedures are illustrated by means of examples. (C) 2002 Elsevier Science B.V. All rights reserved