SHIM and Its Applications

Profiling is the most popular approach to diagnosing performance problems of computer systems. Profiling records run-time system behavior by monitoring software and hardware events either exhaustively or--because of high costs and strong observer effect--periodically. Sampling rates thus determine visibility: the higher the sample rates, the finer-grain behavior observable, and thus the better profilers can help developers analyze and address performance problems. Unfortunately, the sample rates of current profilers are extremely low because of the perturbations generated by their sampling mechanisms. Consequently, current profilers cannot observe insightful fine-grain system behavior. Despite the gigahertz speeds of modern processors, sampling frequencies have been at a standstill--between 1 KHz and 100 KHz--to limit perturbation. This million-cycle gap between two sequential samples blinds profilers to fine-grain behaviors, thus missing root causes of performance problems and potential solutions. My thesis is that by exploiting existing underutilized multicore hardware the sample rates of profilers can be increased by orders of magnitude, leading to new profiling approaches, new discoveries of insightful behavior, and new optimizations. The insights and contributions of this thesis are: 1) We view computer systems as high-frequency signal generators. The high-frequency hardware and software signals that reflect fine-grain system behavior are observable in signal channels: performance counters and shared memory locations. We introduce Shim, a new profiling approach that continuously samples signal channels at resolutions as fine as 15 cycles, which is three to five orders of magnitude finer than current sampling approaches. Shim automatically filters out noisy samples to produce high-fidelity signals. 2) Shim's high-frequency profiling enables a new approach to analyzing and controlling fine-grain system behaviors. We design Tailor, a real-time latency controller for latency-critical web services. Tailor uses a Shim-based high-frequency profiler and an application-level network proxy to continuously monitor and promptly act on the system behaviors that are hazardous to request latency. 3) Shim's fine-grain control of system components enables a new class of online profile-guided optimizations. We introduce Elfen, a Shim-based job scheduler that borrows cycles in short idle periods of latency-critical workloads for batch workloads. Elfen improves CPU utilization significantly without interfering with latency-critical requests by monitoring status changes of latency-critical requests with Shim, and taking real-time scheduling actions. The history of science shows that an order of magnitude or more improvement in measurement fidelity leads to fundamental new discoveries. This thesis fundamentally alters which software and hardware signals are observable on existing systems, and demonstrates that observing these signals stimulates new optimization opportunities