Building Instance Aware Systems using Explicit Performance Modeling

Computer systems are often optimized to realize the best performance possible. However, these optimization techniques are typically blind to the system’s actual workload or the particular environment in which the system is deployed. The lack of awareness of the full picture, or “instance”, limits the extent to which a system’s performance can be improved. However, achieving instance awareness is difficult because it involves optimizing over a large search space of configurable parameters. This thesis explores the use of explicit performance modeling in the context of three different systems to accelerate this optimization process and therefore make instance awareness practical: Flood is a multidimensional database index that is tuned for both lowest latency and minimal space overhead on a query distribution known ahead of time. Flood uses a simple grid-based index system that adjusts the number of partitions in each dimension based on the workload. The “knobs” Flood turns are the parameters of this grid, making it more flexible than existing indexes, which have fewer such knobs. Flood models system performance as a combination of features determined by these grid parameters, and uses explicit measurement to train this model. Cortex is a correlation index that allows databases to index attributes correlated to already-indexed attributes with minimal additional overhead but substantial performance improvement. Cortex decides which points should be considered outliers and inliers in the correlation; this fine grained control is the set of knobs that Cortex needs to instance optimize its performance: for the hardware its on, the host index on the database, and the distribution of queries. Minerva is an end-to-end transport algorithm for video streaming, which aims to achieve Quality-of-Experience fairness, so all clients sharing a bottleneck link in the network have roughly equal picture quality and minimal stalls. Importantly, this is achieved without compromising the bandwidth share of non-video traffic and without any client knowing about the others’ existence. Minerva achieves fairness by making the network instance aware, using information about the client’s state and the videos being streamed to model the client’s performance and scale the aggressiveness of its congestion control algorithm.Ph.D