Measuring and Predicting Computer Software Performance: Tools and Approaches

This thesis consists of material all of which I authored or co-authored: see Statement of Contributions included in the thesis. This is a true copy of the thesis, including any required final revisions, as accepted by my examiners. I understand that my thesis may be made electronically available to the public. ii Measurement-based software performance evaluation is essential to computer science and industry alike, yet despite its widespread adoption, the current level of statistical rigor is inadequate, putting published results into question: for example, the majority of publications fail to report any dispersion metric at all. To foster widespread adoption of statistically rigorous performance evaluation, the first part of this thesis proposes the use of formal experiment design and non-parametric analysis techniques, and presents a distributed infrastructure that lowers the cost of rigorous experimentation by automating the experiment design and execution process, while minimizing the variability in com-puter performance response metrics. Then, to address cases where rigorous performanc