A heuristic method for unstructured pattern management over data streams

Pattern management is an important task in data stream mining and has attracted increasing attention recently. Variations of data stream patterns typically imply some fundamental changes of underlying objects and possess significant domain meanings. Many database applications require investigating the history information to get the knowledge about the evolving process of data streams. However, in most circumstances, the data stream patterns are unstructured: limited memory space cannot record all the patterns discovered online, no training sets or predefined models are available, and large numbers of noises bring another nontrivial challenge. This paper presents our research effort in online pattern management over such streams. A novel algorithm is proposed to detect stream changes, organize meaningful patterns and distinguish useful variations from noises. It extracts new trends from unstructured data heuristically, and involves a special parameter to identify whether the current event should be treated as significant. Several experiments are performed and the results prove this new method feasible and efficient. ? 2010 IEEE.EI