Increasing the effectiveness of technical security testing methods with a comparative study

Organisations use ethical hacking services as a key component to assess their overall cyber security posture. Basis for those services is a set of technical security testing methods such as vulnerability scanning, penetration testing, red teaming, social engineering and similar that are neither clearly and uniformly defined in relevant literature nor have consumers or providers of such methods a common understanding what is and is not part of a certain method. This imposes many problems for both sides: Providers are dealing with consumers requesting a specific security testing method without fully understanding the method itself or its properties. And consumers can request a quote for a specific security testing method and still get no easily comparable basis among different providers. Therefore, the context of this study is this disbalance in knowledge between service providers of ethical hacking services and its consumers about the underlying technical security testing method used for an assessment. To address the issue at hand, this study proposes a security testing landscape model, providing definitions of the most relevant technical security testing methods based on best practice standards, guides and frameworks combined with insights from eight different subject matter experts. Further, the method landscape was enriched with a total of ten properties to characterise the specific nuances of each technical security testing method. This allows a detailed characterisation of each testing method based on its unique properties as well as compare different methods through different properties to aid in selecting a suitable technical security testing method and help reducing the disbalance in knowledge