Algorithms for Distributed Functional Monitoring

Consider the following problem: We have k players each receiving a stream of items, and communicating with a central coordinator. Let the multiset of items received by player i up until time t be A(i)(t). The coordinator's task is to monitor a given function f computed over the union of the inputs boolean OR(i)A(i)(t), continuously at all times t. The goal is to minimize the number of bits communicated between the players and the coordinator. Of interest is the approximate version where the coordinator outputs 1 if f >= tau and 0 if f <= tau (1-epsilon)tau. This defines the (k, f, tau, epsilon) distributed functional monitoring problem. Functional monitoring problems are fundamental in distributed systems, in particular sensor networks, where we must minimize communication; they also connect to the well-studied streaming model and communication complexity. Yet few formal bounds are known for functional monitoring. We give upper and lower bounds for the (k, f, tau, epsilon) problem for some of the basic f's. In particular, we study the frequency moments F(p) for p = 0, 1, 2. For F(0) and F(1), we obtain monitoring algorithms with cost almost the same as algorithms that compute the function for a single instance of time. However, for F(2) the monitoring problem seems to be much harder than computing the function for a single time instance. We give a carefully constructed multiround algorithm that uses "sketch summaries" at multiple levels of details and solves the (k, F(2), tau, epsilon) problem with communication (O) over tilde (k(2)/epsilon + k(3/2)/epsilon(3)). Our algorithmic techniques are likely to be useful for other functional monitoring problems as well