Sources of Variation in Error Sensitivity Measurements, Significant or Not?

Measuring the error sensitivity by fault injection is an important method for assessing the dependability of computer systems. In this paper, we define error sensitivity as the conditional probability that a hardware-related error causes a silent data corruption. When measuring the error sensitivity it is important to consider how the experimental setup and the workload characteristics affect the estimated error sensitivity. We consider five such potential sources of variation (PSVs) in this paper. Three of these are related to the workload: i) input profile, ii) source code implementation, and, iii) use of compiler optimization. Two are related to the experimental setup: i) single vs. double bit-flips, and ii) inject-on-read vs. inject-on-write. The paper discusses the applicability of different statistical tests for assessing whether a PSV has a significant impact on error sensitivity