Developing a systematic approach to evaluate the usability of security APIs

Security Application Programming Interfaces (APIs) play a major role in the software development process. They encapsulate complex security functionalities to provide simple interfaces for programmers who are not experts in computer security. When security APIs are not usable, it leads programmers to make mistakes while developing applications that result in introducing security vulnerabilities into applications. A major reason for the lack of usability of security APIs is that there is no systematic approach to evaluate the usability of security APIs. A systematic approach will allow security API developers to identify usability issues of security APIs and fix them. This will enhance the usability of security APIs and hence, will prevent programmers from making mistakes while using them. This thesis addresses this issue by developing a systematic approach that consists of a set of usability aspects that need to be considered and a set of steps to follow when conducting a security API usability evaluation. By investigating the strengths and weaknesses of different Usability Evaluation Methodologies (UEMs) available for general APIs, this thesis proposed evaluating the usability using a Cognitive Dimensions Framework (CDF) questionnaire as the most suitable UEM to evaluate the usability of security APIs. A four step process was developed in order to conduct a usability evaluation. By reviewing previous literature of security API usability, this thesis further developed a CDF with 15 dimensions that describes usability aspects affecting the usability of security APIs. Thereafter, the developed UEM was evaluated by employing it to identify usability issues for four security APIs and measuring its thoroughness, validity, effectiveness, and reliability. The results of these evaluations indicated that over 80% of the usability issues in a security API can be identified by this methodology with considerably good validity and reliability. Then, a systematic literature review and an empirical evaluation were conducted to improve the data analysis step of the proposed UEM. This step developed a set of guidelines for programmers to follow when performing the data analysis step. The evaluation revealed that the developed set of guidelines provides significant help for evaluators to analyse data collected with the CDF. Based on these results, this thesis contributes to the knowledge by delivering a systematic approach that security API developers can follow to evaluate the usability of security APIs they develop