Achieving High Freshness and Optimal Throughput in Resource-Limited Execution of Multi-Join Continuous Queries

Due to high data volumes and unpredictable arrival rates, continuous query systems processing expensive queries in real-time may fail to keep up with the input data streams - resulting in buffer overflow and uncontrolled loss of data. In this work, we explore join direction adaptation (JDA) to tackle resource-limited processing of multi-join stream queries. While the existing JDA solutions allocate the scarce CPU resources to the most productive half-way join within a single operator, we instead leverage the operator interdependencies to optimize the overall query throughput. We identify result staleness as an impending issue in resource-limited processing, which gets further aggravated if throughput optimizing techniques are employed. For throughput optimization we propose the path- productivity model and further extend it for fulfilling the Freshness tolerance. Our proposed JAQPOT approach is the first integrated solution to achieve near optimal query throughput while also guaranteeing fulfillment of the desired result freshness. JAQPOT runs in quadratic time of the number of streams irrespective of the query plan shape. Our experimental study, using both synthetic and real data sets, demonstrates the superiority of JAQPOT in achieving higher throughput than the state-ofthe- art strategies while, unlike the other methods, also fulfilling freshness predicates