Scalability! but at what cost

We offer a new metric for big data platforms, COST, or the Configuration that Outperforms a Single Thread. The COST of a given platform for a given problem is the hardware configuration required before the platform out-performs a competent single-threaded implementation. COST weighs a system’s scalability against the over-heads introduced by the system, and indicates the actual performance gains of the system, without rewarding sys-tems that bring substantial but parallelizable overheads. We survey measurements of data-parallel systems re-cently reported in SOSP and OSDI, and find that many systems have either a surprisingly large COST, often hundreds of cores, or simply underperform one thread for all of their reported configurations.