Empiriskt test av ett verktyg för sårbarhetsanalys inom IT-säkerhet

This report describes a study aimed at verifying a cyber security modeling language named the Predictive, Probabilistic Cyber Security Modelling Language. This modeling language together with the Enterprise Architecture Analysis Tool acts as a tool for cyber security evaluations of system architectures. To verify the accuracy and readiness of the tool, a generic model of a real life Supervisory Control And Data Acquisition System’s system architecture was modeled using the tool and later evaluated. The evaluation process consisted of a Turing test, which was the same method used for evaluation of the Predictive, Probabilistic Cyber Security Modelling Language predecessor the Cyber Security Modelling Language. For the Turing test, interviews were held with five domain experts within cyber security. Four of which were tasked with creating attack paths given a scenario in the modeled system architecture. The Predictive, Probabilistic Cyber Security Modelling Language was given the same task as the four experts. The attack paths created were consolidated in a standardized form for the last internal company expert within cyber security to evaluate. An expert evaluator was tasked with grading the attack paths produced by the four experts and the Predictive, Probabilistic Cyber Security Modelling Language. The grading was based on how probable the attack paths were perceived by the internal expert. The conclusion was made that given the limitations of the study, the Predictive, Probabilistic Cyber Security Modelling Language produced a cyber security evaluation that was as probable as those created by the human cyber security experts. The results produced were also consistent with the results produced by the Predictive, Probabilistic Cyber Security Modelling Language predecessor the Cyber Security Modelling Language in a previous study. Suggestions for further studies were also introduced which could complement this study and further strengthen the results. This thesis was a collaboration between ABB Enterprise Software and the members of the team behind the Predictive, Probabilistic Cyber Security Modelling Language at ICS at KTH.Denna rapport beskriver en studie vars mål var att verifiera ett modelleringsspråk för datasäkerhet vid namn Predictive, Probabilistic Cyber Security Modelling Language. Detta modelleringsspråk tillsammans med Enterprise Architecture Analysis Tool utgör ett verktyg för datasäkerhetsutvärderingar av systemarkitekturer. För att verifiera exaktheten och mognadsnivån på verktyget så skapades en generisk modell av ett verkligt Supervisory Control And Data Acquisition System-systems arkitektur. Denna modell utvärderades i ett senare skede. Utvärderingsprocessen bestod av ett Turingtest, som är samma metod som användes i en tidigare utvärdering av Predictive, Probabilistic Cyber Security Modelling Languages föregångare Cyber Security Modelling Language. För Turingtestet hölls fem intervjuer med domänexperter inom datasäkerhet. Fyra av dessa fick i uppgift att skapa attackvägar givet ett scenario i den modellerade systemarkitekturen. Attackvägarna som skapades sammanställdes i ett standardiserat formulär för den sista interna företagsexperten inom datasäkerhet att utvärdera. En expertutvärderare fick i uppgift att betygsätta de attackvägar som hade producerats av de fyra experterna och Predictive, Probabilistic Cyber Security Modelling Language. Betygsättningen baserades på hur sannolika de olika attackvägarna uppfattades av den interna experten. Slutsatsen som gjordes var att givet begränsningarna i studien, så producerade Predictive, Probabilistic Cyber Security Modelling Language en datasäkerhetsutvärdering som var likvärdigt sannolik jämfört med de som skapades av mänskliga experter. Resultaten som producerades var också konsistenta med resultaten som producerades av Predictive, Probabilistic Cyber Security Modelling Language föregångare Cyber Security Modelling Language i en tidigare studie. Förslag på kommande studier som skulle komplettera denna studie och stärka resultaten ytterligare introducerades också. Detta examensarbete var ett samarbete mellan ABB Enterprise Software och medlemmarna i teamet bakom Predictive, Probabilistic Cyber Security Modelling Language på ICS på KTH