Stochastic gradient methods for machine learning

Machine learning for “big data” • Large-scale machine learning: large p, large n, large k – p: dimension of each observation (input) – k: number of tasks (dimension of outputs) – n: number of observations • Examples: computer vision, bioinformatics, signal processing • Ideal running-time complexity: O(pn+ kn) – Going back to simple methods – Stochastic gradient methods (Robbins and Monro, 1951) – Mixing statistics and optimization – It is possible to improve on the sublinear convergence rate? Context Machine learning for “big data” • Large-scale machine learning: large p, large n, large k – p: dimension of each observation (input