Tradeoffs for Space, Time, Data and Risk in Unsupervised Learning

Faced with massive data, is it possible to trade off (statistical) risk, and (computational) space and time? This challenge lies at the heart of large-scale machine learning. Using k-means clustering as a prototypical unsupervised learn-ing problem, we show how we can strategi-cally summarize the data (control space) in or-der to trade off risk and time when data is generated by a probabilistic model. Our sum-marization is based on coreset constructions from computational geometry. We also de-velop an algorithm, TRAM, to navigate the space/time/data/risk tradeoff in practice. In par-ticular, we show that for a fixed risk (or data size), as the data size increases (resp. risk in-creases) the running time of TRAM decreases. Our extensive experiments on real data sets demonstrate the existence and practical utility of such tradeoffs, not only for k-means but also for Gaussian Mixture Models.