Modelling the Dynamic Evolution of System Workload During Pipelined Query Execution

Database parallelism increases the complexity of query optimization. In particular, the query optimizer must take into account the impact of processor and network workload on competing and communicating processes. During query execution this additional overhead becomes even more important when exploiting pipelined parallelism, since the workload evolves over time dynamically as processes start and complete execution. We propose a model which incorporates the effect of the system workload on the execution of parallel and pipelined queries. We extend the conventional notion of pipeline in two directions: First, we include pipes with multiple producer nodes into our model. Second, we consider the latency between the start time of the producing process and the start time of the consuming process. Based on this execution model, we derive a cost model that incorporates system workload. In particular, we calculate the effects of pipelining on processor multitasking and the effects of multitas..