Benchmarking the Performance of Application Monitoring Systems

Application-level monitoring of continuously operating software systems provides insights into their dynamic behavior helping to maintain their performance and availability at runtime. Such monitoring may cause a significant runtime overhead to the monitored system depending on the number and location of used instrumentation probes. In order to improve a system's instrumentation to reduce the caused monitoring overhead, it is necessary to know the performance impact of each probe. In this paper, we present our MooBench approach to split the possible causes of monitoring overhead into three portions, and to quantify these portions of monitoring overhead with the help of benchmarks under controlled and repeatable conditions. To the best of our knowledge, most publications on monitoring frameworks provide none or only weak performance evaluations, making comparisons cumbersome. Our benchmark approach provides a basis for such comparisons