Malt: distributed data-parallelism for existing ml applications

Machine learning methods, such as SVM and neural net-works, often improve their accuracy by using models with more parameters trained on large numbers of examples. Building such models on a single machine is often impracti-cal because of the large amount of computation required. We introduce MALT, a machine learning library that inte-grates with existing machine learning software and provides data parallel machine learning. MALT provides abstractions for fine-grained in-memory updates using one-sided RDMA, limiting data movement costs during incremental model up-dates. MALT allows machine learning developers to specify the dataflow and apply communication and representation optimizations. Through its general-purpose API, MALT can be used to provide data-parallelism to existing ML appli-cations written in C++ and Lua and based on SVM, ma-trix factorization and neural networks. In our results, we show MALT provides fault tolerance, network efficiency and speedup to these applications. 1