Balancing memory and performance through selective flushing of software code caches

Dynamic binary translators (DBTs) are becoming increas-ingly important because of their power and flexibility. How-ever, the high memory demands of DBTs present an obsta-cle for all platforms, and especially embedded systems. The memory demand is typically controlled by placing a limit on cached translations and forcing the DBT to flush all trans-lations upon reaching the limit. This solution manifests as a performance inefficiency because many flushed translations require retranslation. Ideally, translations should be selec-tively flushed to minimize retranslations for a given memory limit. However, three obstacles exist: (1) it is difficult to pre-dict which selections will minimize retranslation, (2) selec-tive flushing results in greater book-keeping overheads than full flushing, and (3) the emergence of multicore processors and multi-threaded programming complicates most flushing algorithms. These issues have led to the widespread adop-tion of full flushing as a standard protocol. In this paper, we present a partial flushing approach aimed at reducing retranslation overhead and improving overall performance, given a fixed memory budget. Our technique applies uni-formly to single-threaded and multi-threaded guest applica-tions