Managing Complex Join Queries in Big Data Management Systems

In addition to storing and managing the data and providing capabilities to query them, aDatabase Management System (DBMS) tries to achieve performance goals. High resource utilization, high throughput, and low query execution time are a few of the performance goals that are considered for various DBMSs. The system’s success in achieving its performance goals highly depends on the performance of queries and their operators. Many factors can impact a query’s performance, including how much of its resource requirements are satisfied, when it is scheduled for execution, and which other queries will execute concurrently with it. This thesis is an experimental study focusing on resource management and scheduling techniques to assist a database management system in reaching its performance goals. We begin this thesis by exploring the design space for a robust dynamic Hybrid Hash Join operator, one of the main and most common types of memory-intensive database operators. Our variant of this operator is specifically designed to perform well even when the required statistics and information for a Hybrid Hash Join operator are unavailable or inaccurate. Next, we explore various memory management and execution strategies for efficiently executing queries containing multiple join operators. We specifically study variations of Left Deep Trees, Right Deep Trees, and Bushy Trees containing one to eight join operators. We evaluate their performance under different memory availabilities, join and scan selectivities, degrees of parallelism, storage types, and query complexities. Lastly, we study and evaluate the performance of various schedulers designed to schedule queries with highly different memory requirements and execution times in a concurrent environment. Our performance goal is to design a fair scheduler that keeps different classes of queries in admission and resource control queues in proportion to their execution times