Perpetual adaptation of software to hardware : an extensible architecture for providing code optimization as a central system service

Much of the software in everyday operation is not making optimal use of the hardware on which it actually runs, but has been optimized for earlier and often outdated processor versions. This is because users upgrade hardware independently from, and often more frequently than, application software. Moreover, software vendors are generally unable or unwilling to provide multiple versions of the same program that differ only in the processor model specifically targeted during compilation.The obvious solution to matching a piece of softWcU'e with the actual capabilities of the hardware on which it is about to be executed is to delay code generation until load time. This is the earliest point at which the software can be fine-tuned to specific hardware characteristics such as the latencies of individual instructions and the sizes of the instruction and data caches. An even better match can be achieved by replacing the already executing software at regular intervals by new versions constructed on-the-fly using a background code re-optimizer. The code produced using such dynamic re-optimizers is often of a higher quality than can be achieved using static "off-line" compilation because live profiling data can be used to guide optimization decisions and the software can hence adapt to changing usage patterns and the late addition of dynamic link libraries. [...