A Self-Tuning Disk Striping System for Parallel Input/Output

In this thesis, we propose a self-tuning approach for automatically selecting and refining the file system's striping parameters based on application access patterns. The technique presented here relies on the monitoring of application I/O requests and the application of an analytic model to determine file striping parameters that improve overall file system performance. The application I/O requests are characterized by their size, type, duration and inter-arrival times, and clustered into representative groups to determine typical request sizes. Then, an analytical striping model is used to determine potential performance improvements. Input parameters to this model include the disk and network system characteristics, and knowledge of application I/O access pattern behavior. The output of the model is the estimated response time as a function of the request size, request rate and stripe depth. An optimum stripe depth that minimizes the average response time can be calculated from this output. Two file restriping methods are studied, and a 3-competitive online algorithm for determining when to restripe is presented. Finally, we present results gathered from executing three high-performance scientific applications, WaveToy, sPPM and Montage, on three different computer clusters. Our results show that I/O performance for the WaveToy and sPPM applications can be improved significantly by striping their output files across several disks using a stripe depth derived from applying the restriping model to the application's I/O access patterns