Local Correlation Detection with Linearity Enhancement in Streaming Data

This paper addresses the challenges in detecting the potential cor-relation between numerical data streams, which facilitates the re-search of data stream mining and pattern discovery. We focus on local correlation with delay, which may occur in burst at different time in different streams, and last for a limited period. The un-certainty on the correlation occurrence and the time delay make it diff cult to monitor the correlation online. Furthermore, the con-ventional correlation measure lacks the ability of ref ecting visual linearity, which is more desirable in reality. This paper proposes effective methods to continuously detect the correlation between data streams. Our approach is based on the Discrete Fourier Trans-form to make rapid cross-correlation calculation with time delay allowed. In addition, we introduce a shape-based similarity mea-sure into the framework, which ref nes the results by representative trend patterns to enhance the signif cance of linearity. The similar-ity of proposed linear representations can quickly estimate the cor-relation, and the window sliding strategy in segment level improves the eff ciency for online detection. The empirical study demon-strates the accuracy of our detection approach, as well as more than 30 % improvement of eff ciency