Efficient Mining of Statistical Dependencies

The Multi-Stream Dependency Detection algorithm finds rules that capture statistical dependencies between patterns in multivariate time series of categorical data [ Oates and Cohen, 1996c ] . Rule strength is measured by the G statistic [ Wickens, 1989 ] , and an upper bound on the value of G for the descendants of a node allows msdd's search space to be pruned. However, in the worst case, the algorithm will explore exponentially many rules. This paper presents and empirically evaluates two ways of addressing this problem. The first is a set of three methods for reducing the size of msdd's search space based on information collected during the search process. Second, we discuss an implementation of msdd that distributes its computations over multiple machines on a network