A transparent runtime data distribution engine for OpenMP1

This paper makes two important contributions. First, the pa-per investigates the performance implications of data place-ment in OpenMP programs running on modern NUMA mul-tiprocessors. Data locality and minimization of the rate of re-mote memory accesses are critical for sustaining high perfor-mance on these systems. We show that due to the low remote-to-local memory access latency ratio of contemporary NUMA architectures, reasonably balanced page placement schemes, such as round-robin or random distribution, incur modest per-formance losses. Second, the paper presents a transparent, user-level page migration engine with an ability to gain back any performance loss that stems from suboptimal placement of pages in iterative OpenMP programs. The main body of the paper describes how our OpenMP runtime environment uses page migration for implementing implicit data distribu-tion and redistribution schemes without programmer inter-vention. Our experimental results verify the effectiveness of the proposed framework and provide a proof of concept that it is not necessary to introduce data distribution directives in OpenMP and warrant the simplicity or the portability of the programming model. ∗Corresponding author. 1An earlier version of this paper [29] appeared in the IEEE/ACM Supercompting’2000 Conference and won the Best Technical Paper Award. 1