Using Hybrid Parallelism to Improve Memory Use in the Uintah Framework

The Uintah Software framework was developed to provide an envi-ronment for solving fluid-structure interaction problems on struc-tured adaptive grids on large-scale, long-running, data-intensive problems. Uintah uses a combination of fluid-flow solvers and particle-based methods for solids together with a novel asynchronous task-based approach with fully automated load balancing. Uintah’s memory use associated with ghost cells and global meta-data has become a barrier to scalability beyond O(100K) cores. A hybrid memory approach that addresses this issue is described and evalu-ated. The new approach based on a combination of Pthreads and MPI is shown to greatly reduce memory usage as predicted by a simple theoretical model, with comparable CPU performance. Categories and Subject Descriptor