PAL: Propagation-aware anomaly localization for cloud hosted distributed applications

Distributed applications running inside cloud are prone to performance anomalies due to various reasons such as insuf-ficient resource allocations, unexpected workload increases, or software bugs. However, those applications often consist of multiple interacting components where one component anomaly may cause its dependent components to exhibit anomalous behavior as well. It is challenging to identify the faulty components among numerous distributed application components. In this paper, we present a Propagation-aware Anomaly Localization (PAL) system that can pinpoint the source faulty components in distributed applications by ex-tracting anomaly propagation patterns. PAL provides a robust critical change point discovery algorithm to accu-rately capture the onset of anomaly symptoms at different application components. We then derive the propagation pattern by sorting all critical change points in chronologi-cal order. PAL is completely application-agnostic and non-intrusive, which only relies on system-level metrics. We have implemented PAL on top of the Xen platform and tested it on a production cloud computing infrastructure using the RUBiS online auction benchmark application and the IBM System S data streaming processing application with a range of common software bugs. Our experimental results show that PAL can pinpoint faulty components in distributed applications with high accuracy and low overhead