Learning with Matrix Factorizations

Matrices that can be factored into a product of two simpler matricescan serve as a useful and often natural model in the analysis oftabulated or high-dimensional data. Models based on matrixfactorization (Factor Analysis, PCA) have been extensively used instatistical analysis and machine learning for over a century, withmany new formulations and models suggested in recent years (LatentSemantic Indexing, Aspect Models, Probabilistic PCA, Exponential PCA,Non-Negative Matrix Factorization and others). In this thesis weaddress several issues related to learning with matrix factorizations:we study the asymptotic behavior and generalization ability ofexisting methods, suggest new optimization methods, and present anovel maximum-margin high-dimensional matrix factorizationformulation