Optimized Data Placement for Real-time Analytics on Semi-structured Data

In this dissertation, we address the emerging demand for extending traditional relational support to semi-structured data, and for real-time data analytics. Semi-structured data (e.g.,JSON) poses new challenges with large numbers of attributes, sparse attributes and dynamic changes in both workload and data set. In this context, we design, implement and evaluate a novel technique for vertical partitioning of data for optimizing performance of main memory databases. Our partitioning algorithm enables JSON data storage in main memory relational databases by intelligently decomposing JSON objects into different tables. It also adapts to changes in workload and dataset by dynamically refining the current layout. Using the Nobench dataset for JSON data, we outperform Argo, a state-of-the-art data model that maps JSON into relational databases, by 15-30x. We also outperform Hyrise, a state-of-the-art vertical partitioning algorithm, by 1.6x for hybrid workloads with sparse data and perform as well as Hyrise for other workloads.M.A.S