Big Data Bites

W hat’s growing faster, Big Data or Moore’s Law?It’s undeniable that the amount of data that organizations must store, process, organize, and analyze is growing rapidly. This requires increasingly larger clusters and data-centers, as well as increasingly complex software infrastructure to orchestrate the necessary com-putations. But is Big Data growing faster than Moore’s Law is lowering the costs of computing capabilities to accomplish these tasks? For rhetor-ical convenience, I’m using Big Data to refer to all the things we want to do on massive collections of data, and Moore’s Law to refer to exponen-tial increases in computing capabilities for doing it. It’s worth emphasizing that I don’t literally mean the periodic doubling of transistors on a chip; I use Moore’s Law as a convenient short-hand to refer to continued exponential advances in computing. Logically, there are only three possibilities: 1. Big Data is growing faster than Moore’s Law. 2. Big Data is growing at the same rate a