MODEL-BASED STRESS TESTING FOR DATABASE SYSTEMS

Database Management Systems (DBMS) have been successful at processing transaction workloads over decades. But contemporary systems, including Cloud computing, Internet-based systems, and sensors (i.e., Internet of Things (IoT)), are challenging the architecture of the DBMS with burgeoning transaction workloads. The direct consequence is that the development agenda of the DBMS is now heavily concerned about meeting non-functional requirements, such as performance, robustness and scalability. Otherwise, any stressing workload will make the DBMS lose control of simple functional requirements, such as responding to a transaction request~\cite{stem}. While traditional DBMS, including DB2, Oracle, and PostgreSQL, require embedding new features to meet non-functional requirements, the contemporary DBMS called as NewSQL present a completely new architecture. What is still lacking in the development agenda is a proper testing approach coupled with burgeoning transaction workloads for validating the DBMS with non-functional requirements in mind. The typical non-functional validation is carried out by performance benchmarks. However, they focus on metrics comparison instead of finding defects. In this thesis, we address this lack by presenting different contributions for the domain of DBMS stress testing. These contributions fit different testing objectives to challenge each specific architecture of traditional and contemporary DBMS. For instance, testing the earlier DBMS (e.g., DB2, Oracle) require incremental performance tuning (i.e., from simple setup to complex one), while testing the latter DBMS (e.g., VoltDB, NuoDB) require driving it into different performance states due to its self-tuning capabilities. Overall, this thesis makes the following contributions: 1) Stress TEsting Methodology (STEM): A methodology to capture performance degradation and expose system defects in the internal code due to the combination of a stress workload and mistuning; 2) Model-based Database Stress Testing (MoDaST): An approach to test NewSQL database systems. Supported by a Database State Machine (DSM), MoDaST infers internal states of the database based on performance observations under different workload levels; 3) Under Pressure Benchmark (UPB): A benchmark to assess the impact of availability mechanisms in NewSQL database systems. We validate our contributions with several popular DBMS. Among the outcomes, we highlight that our methodologies succeed in driving the DBMS up to stress state conditions and expose several related defects, including a new major defect in a popular NewSQL