The Cell Probe Complexity of Succinct Data Structures

In the cell probe model with word size 1 (the bit probe model), a static data structure problem is given by a map f : {0,1}^n * {0,1}^m -> {0,1}, where {0,1}^n is a set of possible data to be stored, {0,1}^m is a set of possible queries (for natural problems, we have m {0,1}^s and a query algorithm q so that q(phi(x), y) = f(x,y). The time t of the query algorithm is the number of bits it reads in phi(x). In this paper, we consider the case of succinct representations where s = n + r for some redundancy r = Omega(n/log n). In particular, for very small redundancies r, we get an almost optimal lower bound stating that the query algorithm has to inspect almost the entire data structure (up to a logarithmic factor). We show similar lower bounds for problems satisfying a certain combinatorial property of a coding theoretic flavor. Previously, no omega(m) lower bounds were known on t in the general model for explicit functions, even for very small redundancies. By restricting our attention to systematic or index structures phi satisfying phi(x) = x · phi*(x) for some map phi* (where · denotes concatenation) we show similar lower bounds on the redundancy-query time trade-off for the natural data structuring problems of Prefix Sum and Substring Search