Pgg: An online pattern based approach for stream variation management

Abstract Many database applications require efficient processing data streams with value variations and fluctuant sampling frequency. The variations typically imply fundamental features of the stream and im-portant domain knowledge for the underlying objects. In some data streams, successive events seem to recur in a certain time interval, but the data indeed evolves with tiny differences as time elapses. This fea-ture, so called pseudo periodicity, poses a new challenge to stream variation management. This study fo-cuses on the online management for variations over such streams. The idea can be applied to many sce-narios such as patient vital signal monitoring in medical applications. This paper proposes a new method named Pattern Growth Graph (PGG) to detect and manage variations over evolving streams with follow-ing features: 1) adopts the wave-pattern to capture the major information of data evolution and represent them compactly; 2) detects the variations in a single pass over the stream with the help of wave-pattern matching algorithm; 3) only stores different segments of the pattern for incoming stream, and hence sub-stantially compress the data without losing important information; 4) distinguishes meaningful data changes from noise and reconstructs the stream with acceptable accuracy. Extensive experiments on real datasets containing millions of data items, as well as a prototype system, are carried out to demonstrate the feasibility and effectiveness of the proposed scheme