JSONiq and RumbleDB on Snowflake

This thesis integrates Snowflake to the ecosystem of RumbleDB, the query execution engine of the JSONiq query language for large, messy, and heterogeneous data. This enables RumbleDB to take advantage of major benefits provided by cloud-native database services such as usability, scalability, elasticity, and productivity, without performance compromise. Data independence achieved by RumbleDB make this integration possible. We only update the logic of the actual JSONiq query execution in the implementation of RumbleDB’s runtime iterators. Snowflake’s Snowpark APIs are utilized tomapJSONiq expressions to Snowflake SQL queries for querying data in the Snowflake data cloud. We add a Snowflake execution mode for runtime iterators in RumbleDB with three execution formats, so thatwe can maximally exploit Snowflake’s native support on semi-structured and heterogeneous data. We implement all the 13 JSONiq expressions that can be parallelized and the core functionalities of all seven clauses of FLWOR expressions with Snowflake. Performance experiments show that our implementation of RumbleDB on Snowflake has a promising performance. Our implementation significantly out-performs a locally running RumbleDB even when the queried data is smaller than 100MB. Our implementation is also well-optimized, since the performance of a JSONiq query can be only 10% slower than the performance of its handwritten Snowflake SQL counterpart. We also achieved decent scalability: we can process TBs of data with RumbleDB on Snowflake, and the runtime of the same query can successfully drop by nearly 2x if users double the computing power after resource saturation