Automated, Requirements-based Security Testing of Web-oriented Software Systems

Motivation and Context. Modern Internet-based services (e.g., home-banking, personal-training, healthcare) are delivered through Web-oriented software systems which run on multiple and different devices including computers, mobile devices, wearable devices, and smart TVs. They manage and exchange users’ personal data such as credit reports, locations, and health status. Therefore, the security of the system and its data are of crucial importance. Unfortunately, from security requirements elicitation to security testing, there are a number of challenges to be addressed to ensure the security of Web-oriented software systems. First, existing practices for capturing security requirements do not rely on templates that ensure the specification of requirements in a precise, structured, and unambiguous manner. Second, security testing is usually performed either manually or is only partially automated. Most of existing security testing automation approaches focus only on specific vulnerabilities (e.g., buffer overflow, code injection). In addition, they suffer from the oracle problem, i.e., they cannot determine that the software does not meet its security requirements, except when it leads to denial of service or crashes. For this reason, security test automation is usually partial and only addresses the generation of inputs and not the verification of outputs. Though, in principle, solutions for the automated verification of functional requirements might be adopted to automatically verify security requirements, a number of concerns remain to be addressed. First, there is a lack of studies that demonstrate their applicability, in the context of security testing. Second, the oracle problem remains an open problem in many aspects of software testing research, not only security testing. In the context of functional testing, metamorphic testing has shown to be a viable solution to address the oracle problem; however, it has never been studied in the context of security testing. Contributions. In this dissertation, we propose a set of approaches to address the above-mentioned challenges. (1) To model security requirements in a structured and analyzable manner, we propose a use case modeling approach that relies on a restricted natural language and a template already validated in the context of functional testing. It introduces the concepts of security use case specifications (i.e., what the system is supposed to do) and misuse case specifications (i.e., malicious user behaviours that the system is supposed to prevent). Moreover, we propose a template for capturing guidelines for the mitigation of security threats. (2) To verify that systems meet their security requirements, we propose an approach to automatically generate security test cases from misuse use case specifications. More precisely, we propose a natural language programming solution that automatically generates executable security test cases and test inputs from misuse case specifications in natural language. (3) To address the oracle problem, we propose a metamorphic testing solution for Web-oriented software systems. The solution relies on a predefined set of metamorphic relations that capture (a) how an attacker likely alters a valid input to exploit a vulnerable system and (b) how the output of the system should change as a result of the attack if the system meet its security requirements. Our solution relies on Web-crawlers to automatically identify the valid inputs to be used for testing. (4) We identify a set of testability guidelines to facilitate the adoption of the proposed approaches in software projects. The identified guidelines indicate (a) which types of vulnerabilities can be addressed through the solutions proposed in this dissertation and (b) which design solutions should be integrated into the system to enable effective test automation