Slider: Incremental Sliding-Window Computations for Large-Scale Data Analysis," CITI, Universidade Nova de

Sliding-window computations are widely used for data anal-ysis in networked systems. Such computations can consume significant computational resources, particularly in live sys-tems, where new data arrives continuously. This is because they typically require a complete re-computation over the full window of data every time the window slides. There-fore, sliding-window computations face important scalabil-ity problems. In this paper, we propose techniques for im-proving the scalability by performing sliding-window com-putations incrementally. In this paradigm, when some new data is added at the end of the window or old data dropped from its beginning, the output is updated efficiently by reusing previously run sub-computations, avoiding a complete re-computation. To realize this approach, we propose Slider, a novel framework that supports incremental sliding-window computations transparently and efficiently by leveraging self-adjusting computations principles of dynamic dependence graphs and change propagation. We implemented Slider based on the Hadoop MapReduce framework with a declarative SQL-like query interface, and evaluated it with a variety of applications and real world case studies from networked system. Our results show significant performance improve-ments for large-scale sliding-window computations without any modifications to the existing application code. 1