A Study of an In-memory Database System for Real-time Analytics on Semi-structured Data Streams

Recently there have been increasing demands for real-time analytics on rapidly changing data and for databases to effectively support mixed OLTP and OLAP workloads. In-memory databases provide a promising paradigm for these applications. However, due to the rapid emergence of various types of semi-structured data, one key challenge for in-memory databases is the data layout. In this dissertation, we developed an in-memory database system that dynamically partitions its tables vertically based on workload characteristics to achieve fast querying speed on semi-structured data. We produced a set of guidelines on vertical partitioning in-memory data in different situations and showed that our approach can outperform traditional columnar and row-based storage methods, as well as an alternative data structure, ARGO, that was recently developed to enable JSON storage in relational databases. We also showed that our system has advantages over a system with similar idea, called Hyrise, in partitioning adaptability.M.A.S