Semantic-aware Anomaly Detection in Distributed Storage Systems

As distributed storage systems become central to business operations, increasing their reliability becomes an essential requirement. Towards this goal, distributed storage systems come equipped with various monitoring probes. As a result, they generate massive amounts of monitoring data. The amount and complexity of such monitoring data surpasses the human ability to detect anomalies in this data. Our thesis is that, in order to obtain a practical solution to the reliability issues posed by modern distributed storage systems, the semantics of these systems must be incorporated into the anomaly detection process. Our semantic-aware approach enables digesting large amounts of monitoring data, through statistical methods, into a few human understandable patterns. These selected unusual patterns provide intuitive pinpointing towards the possible root causes of an anomaly; they can be leveraged by the human in the loop for root-cause anomaly analysis and related investigations. Our strengths lie in the lightweight, minimal overhead of our techniques, as well as their sensitivity to detect even subtle and localized unusual patterns. In this dissertation, we introduce three semantic-aware anomaly detection techniques: i) anomaly detection based on validation of high-level invariants, ii) stage-aware anomaly detection through runtime tracking of execution flows inferred from existing log statements in the code, and iii) a composite anomaly detection model using cross-component correlations. We studied our methods in two realistic environments: networked multi-tier storage systems used in enterprise settings, and a variety of scalable distributed storage platforms used in large data centers, i.e., HBase, HDFS and Cassandra. Our evaluation shows that our methods are lightweight, introducing negligible overhead, and detect anomalies in real-time.Ph