Understanding bottom-up and top-down visual attention mechanisms related to visual quality perception can be greatly beneficial for the design of effective objective quality metrics. Subjective studies based on eye-movement tracking have been recently published that try to get more insight in these interactions. However, it is still not easy to find coherence across their results, also due to the different methodologies adopted to analyze eye-tracking data. In this paper we propose a robust methodology to measure differences between eye-tracking data collected under different experimental conditions. The proposed method takes into account inter-observer variability and content effects, producing results that give an accurate insight in atte...