and its Use for the Study of

We present a suite Df tools- the Musketeers- for monitoring and analysis of paging, I/O and communication activity of parallel programs. The tools support automatic instru-mentation and allow the user to select the statements to be instrumented and to define the format of a trace record, and to include data from kernel structures and from the user's address space. We discuss a methodology to correlate the activity of processes running concurrently, regardless of the size of the prOCess group. Such a correlation is necessary because I/O and paging devices, as well as the interconnection network of an MPP, are shared resources subject to the load generated by all processes running concurrently. In addition to finding the load placed upon system resources, the methodology is useful for studying means to improve the I/O throughput- by buffering I/O requests directed to the same device within a certain time window and transferring large I/O data blocks from the I/O device- Or for studying scheduling. Finally, we present results of measurements of paging and I/O activity for several parallel programs