Program Transformations for Vulnerability Detection in Binary Executable Files

The major causes of threats in computer systems across the globe have been attributed to vulnerabilities in the underlying software on any hardware platform. Vulnerability analysis entails the process of determining whether a system contains a flaw which an attacker can exploit to compromise the system. Across the years, many approaches and strategies to perform vulnerability analysis and detection have been developed for high-level programming languages but little research has been carried out to detect vulnerability in binary executable codes. This remains a major challenge that must be addressed in the computer science community. This is important given that the first point of attack by hackers is through the binary data level. It is the level through which a potential hacker establishes security loopholes and secret doorways that are not easily detected in low-level programming languages. As such, in an effort to curb the threats from vulnerability, software security developers require better and improved methods and models to analyse binary executable files. Currently, the type of vulnerability tools available to software developers remains fragmented and cannot perform detailed analysis with the desired level of accuracy. In order to aid the detection of vulnerabilities in low-level language by software developers, this research proposes a novel systematic approach for detecting vulnerabilities at the binary level. The approach taken was based on static vulnerability analysis, which entails the re-engineering of low level language by leveraging on the abundant transformation techniques such as FermaT transformation system within Wide Spectrum Language (WSL) program. This research highlights a novel approach and contribution for the extension of existing WSL to enhance its vulnerability detection capabilities using two techniques namely: (i) program slicing combined with static taint analysis and (ii) static single assignment combined with value range analysis. The developed model provides software developers with a new approach of vulnerability detection mechanism in binary executable files. The usefulness of the approach was tested using several scenarios to validate its output and a high level accuracy in the detection pattern was observed. The results show that possible vulnerabilities can be detected in binary executable files