Gaussian Process Modeling and Supervised Dimensionality Reduction Algorithms via Stiefel Manifold Learning

Much research has gone into scaling up classical machine learning algorithms such as Gaussian Processes (GPs), but the curse of dimensionality still remains. While many supervised dimensionality reduction algorithms have been proposed in the literature, few of them can scale up to large data-sets. Furthermore, the majority of dimensionality reduction techniques are tailored for classication problems, which leaves regression tasks unexplored. The contributions of this thesis are threefold. First, we extend classical active subspace (AS) theory to a non-linear counterpart. Secondly, we introduce a scalable non linear supervised principal component analysis (SPCA) algorithm. Thirdly, we propose a novel class of supervised dimensionality reduction algorithms called smoothening analysis (SA). The SA algorithms consist of scalable linear and non-linear frequentist and Bayesian algorithms tailored (but not limited) for regression tasks that learn low dimensional Stiefel manifolds.M.A.S