An empirical evaluation of the effectiveness of attack graphs and fault trees in cyber-attack perception

Perceiving and understanding cyber-attacks can be a difficult task. This problem is widely recognised and well documented, and more effective techniques are needed to aid cyber-attack perception. Attack modelling techniques (AMTs) –such as attack graphs and fault trees, are useful visual aids that can aid cyber-attack perception; however, there is little empirical or comparative research which evaluates the effectiveness of these methods. This paper reports the results of an empirical evaluation between an adapted attack graph method and the fault tree standard to determine which of the two methods is more effective in aiding cyber-attack perception. An empirical evaluation (n=63) was conducted through a 3×2×2 factorial design. Participants from computer-science and non computer-science backgrounds were divided into an adapted attack graph and fault tree group and then asked to complete three tests which tested the ability to recall, comprehend and apply the attack modelling technique. A mean assessment score (mas) was calculated for each test. The results show that the adapted attack graph method is more effective at aiding cyber-attack perception when compared with the fault tree method (p <0.01). Participants that have a computer science background outperformed other participants when using both methods (p