Processing flows of information: From data stream to complex event processing

A large number of distributed applications requires continuous and timely processing of informa-tion as it flows from the periphery to the center of the system. Examples are intrusion detection systems, which analyze network traffic in real-time to identify possible attacks; environmental monitoring applications, which process raw data coming from sensor networks to identify criti-cal situations; or applications performing on-line analysis of stock prices to identify trends and forecast future values. Traditional DBMSs, which need to store and index data before processing it, can hardly fulfill the requirements of timeliness coming from such domains. Accordingly, during the last decade different research communities developed a number of tools, which we collectively call Informa-tion Flow Processing (IFP) Systems, to support these scenarios. They differ in their system architecture, data model, rule model, and rule language. In this paper we survey these systems to help researchers, often coming from different backgrounds, in understanding how the various approaches they adopt may complement each other. In particular, we propose a general, unifying model to capture the different aspects of an IFP system and use it to provide a complete and precise classification of the systems and mechanisms proposed so far