Monet: a next-generation database kernel for query-intensive applications

htmlabstractMonet is a database kernel targeted at query-intensive, heavy analysis applications (the opposite of transaction processing), which include OLAP and data mining, but also go beyond the business domain in GIS processing, multi-media retrieval and XML. The clean sheet approach of Monet tries to depart from the traditional RDBMS design and implementation patterns in an attempt to obtain best performance on modern hardware, which has changed a lot since the currently dominant relational database systems were designed and developed. While most hardware components have experienced exponential growth in power over the years (a.k.a. Moore's law), I/O and especially memory latency have been lagging, creating an exponentially growing bottleneck. Additionally, modern hyperpipelined CPUs increasingly require code that is fully predictable (as to avoid branch mispredictions) and independent (to exploit parallel execution units) in order to reach their advertised performance, which makes a tough match with the interpreted, highly unpredictable and interdependent code found in database execution engines. The choice in Monet for the Decomposed Storage Model (DSM), which stores data in binary (2-column) tables only is motivated by the fact that query-intensive access patterns often profit from a vertically fragmented physical data model, which minimizes both I/O and cache misses when queries touch many rows but few columns. The column-wise processing model followed in the MIL language allows for a ``RISC'' (Reduced Instruction Set) query processing algebra, whose operators have a very low degree of freedom, thus allowing for a CPU-wise highly efficient implementation (i.e. one that consists of predictable and independent instructions). Also, specific attention was paid in Monet in developing cache-conscious query processing algorithms, in particular the radix-algorithms for join processing. This thesis is a reference to the Monet system in all its detail, and also outlines an SQL front-end that uses Monet as a back-end, for constructing a full-fledged SQL compliant RDBMS including ACID properties