Achieving high performance in a LBM code using OpenMP

This paper describes how specific architectural and implementation aspects of an SMP system (in the present work, an IBM POWER3 16-way node) influence performance and scalability of a very simple but real world code. The OpenMP parallelization of a Lattice Boltzmann Method code is presented. Scaling up to more CPUs and bigger grids successively elicits limitations arising from the hardware implementations. A sequence of substantial changes to the original implementation, aimed to overcome those obstacles, is detailed. The case study shows that an SMP system cannot be simply modelled as a bunch of processors sharing memory: making a big memory area available to each CPU in a single application can cause some processor implementation limits, unnoticed in usual serial programs, to come to evidence.