SCALABLE PROCESSING OF MULTIPLE AGGREGATE CONTINUOUS QUERIES

Data Stream Management Systems (DSMSs) were developed to be at the heart of every monitor- ing application. Monitoring applications typically register hundreds of Continuous Queries (CQs) in DSMSs in order to continuously process unbounded data streams to detect events of interest. DSMSs must be designed to efficiently handle unbounded streams with large volumes of data and large numbers of CQs, i.e., exhibit scalability. This need for scalability means that the underlying processing techniques a DSMS adopts should be optimized for high throughput (i.e., tuple output rate). Towards this, two main approaches were proposed in the literature: (1) Multiple Query Opti- mization (MQO) and (2) Scheduling. In this dissertation we focus on optimizing the processing of multiple Aggregate Continuous Queries (ACQs), given their high processing cost and popularity in all monitoring applications.\ud Specifically, in this dissertation, we explore shared processing of ACQs and introduce the con- cept of ’Weaveability’ as an indicator of the potential gains of sharing the processing of ACQs. We develop Weave Share, a multiple ACQs optimizer that considers the different uncorrelated factors of the processing cost, such as the input rate and ACQs’ specifications. In order to fully reap the benefits of the new weave-based optimization techniques, we conceptualize a new underlying ag- gregate operator implementation and realize it in the TriOps framework. TriOps enables adaptive sharing of multiple ACQs that have different window specification, predicates and group-by at- tributes. The properties of the proposed techniques are studied analytically and their performance advantages are experimentally evaluated using simulation and in the context of the AQSIOS DSMS prototype