Data-Driven Dimensionality Reduction: Manifold Learning for Optimization

With rapidly expanding volumes of data across all quantitative disciplines, there is a great need for computational methods that can overcome the curse of dimensionality when working with an enormous number of inputs or parameters. This work explores data-driven strategies for discovering and exploiting two types of structure by which we can view high-dimensional optimization problems in terms of fewer effective inputs: parameter non-identifiability and multiscale potentials. In the instance of parameter non-identifiability, the objective’s dependence on its inputs can be expressed as a function of relatively few combinations of parameters (e.g., the Reynolds number in fluid mechanics, though we are not limited to nondimensionalized quantities). In the case of multiscale potentials, the objective value changes at a substantially greater rate in some directions than in others, so the dynamics of typical optimization algorithms are quickly attracted to a low-dimensional manifold comprising the directions of slower variation. In both circumstances, our primary tool for dimensionality reduction is diffusion maps (DMaps), a manifold learning technique that computes a parameterization of the lower-dimensional surface on which an ensemble of sample points approximately lies. If one can discover this manifold within the ambient space that contains the original data, then modeling the objective in terms of a reduced set of DMaps coordinates lessens the computational expense of locating a solution. We consider examples designed to demonstrate the proposed frameworks and present collaborative work in applications toward zeolite fabrication and the design of high-entropy alloy catalysts