A Tracing Environment for MPI

Event tracing of parallel programs can provide valuable information about program performance. The design of MPI provides a profiling interface for inserting event tracing. This paper will discuss how some of the functionality of the instrumentation package from PICL was incorporated into a profiling environment for MPI. The PICL routines necessary to support the creation of tracefiles have been integrated into this system. The tracefiles generated follow the (new) PICL trace format and can be displayed using ParaGraph or any other tool that uses this format. This new profiling system potentially allows for the tracing of all MPI routines. However, MPI has many more options and variations than PICL, so some of the MPI routines need to have trace formats defined for them. To help control the amount of output as well as the potential perturbation of execution, the user can specify the level of tracing desired. We also discuss the analysis of the tracefiles to adjust timestamps for clock ..