Constraint Methods for Neural Networks and Computer Graphics

Both computer graphics and neural networks are related, in that they model natural phenomena. Physically-based models are used by computer graphics researchers to create realistic, natural animation, and neural models are used by neural network researchers to create new algorithms or new circuits. To exploit successfully these graphical and neural models, engineers want models that fulfill designer-specified goals. These goals are converted into mathematical constraints. This thesis presents constraint methods for computer graphics and neural networks. The mathematical constraint methods modify the differential equations that govern the neural or physically based models. The constraint methods gradually enforce the constraints exactly. This thesis also describes applications of constrained models to real problems. The first half of this thesis discusses constrained neural networks. The desired models and goals are often converted into constrained optimization problems. These optimization problems are solved using first-order differential equations. There are a series of constraint methods which are applicable to optimization using differential equations: the Penalty Method adds extra terms to the optimization function which penalize violations of constraints, the Differential Multiplier Method adds subsidiary differential equations which estimate Lagrange multipliers to fulfill the constraints gradually and exactly, Rate-Controlled Constraints compute extra terms for the differential equation that force the system to fulfill the constraints exponentially. The applications of constrained neural networks include the creation of constrained circuits, error-correcting codes, symmetric edge detection for computer vision, and heuristics for the traveling salesman problem. The second half of this thesis discusses constrained computer graphics models. In computer graphics, the desired models and goals become constrained mechanical systems, which are typically simulated with second-order differential equations. The Penalty Method adds springs to the mechanical system to penalize violations of the constraints. Rate-Controlled Constraints add forces and impulses to the mechanical system to fulfill the constraints with critically damped motion. Constrained computer graphics models can be used to make deformable physically-based models follow the directives of a animator.