Shadow profiling: Hiding instrumentation costs with parallelism

In profiling, a tradeoff exists between information and overhead. For example, hardware-sampling profilers incur negligible overhead, but the information they collect is con-sequently very coarse. Other profilers use instrumentation tools to gather temporal traces such as path profiles and hot memory streams, but they have high overhead. Runtime and feedback-directed compilation systems need detailed infor-mation to aggressively optimize, but the cost of gathering profiles can outweigh the benefits. Shadow profiling is a novel method for sampling long traces of instrumented code in parallel with normal execution, taking advantage of the trend of increasing numbers of cores. Each instrumented sample can be many millions of instructions in length. The primary goal is to incur negligible overhead, yet attain pro-file information that is nearly as accurate as a perfect pro-file. The profiler requires no modifications to the operating system or hardware, and is tunable to allow for greater coverage or lower overhead. We evaluate the performance and accuracy of this new profiling technique for two com-mon types of instrumentation-based profiles: interprocedu-ral path profiling and value profiling. Overall, profiles col-lected using the shadow profiling framework are 94 % accu-rate versus perfect value profiles, while incurring less than 1 % overhead. Consequently, this technique increases the vi-ability of dynamic and continuous optimization systems by hiding the high overhead of instrumentation and enabling the online collection of many types of profiles that were pre-viously too costly.