Automated Testing for Solidity smart contracts

Blockchains are the underlying technology for making secure online transactions using cryptocurrencies such as Bitcoins and Ethers. Executing, verifying, and enforcing credible transactions on permissionless blockchains is done using smart contracts. Smart contracts are publicly available programs that run on the Ethereum network and can be implemented by Solidity language. Solidity is the most popular programming language to implement smart contracts. Smart contracts help exchange money, property, shares or anything of value in a transparent, conflict-free way while reducing transaction costs associated with third-party contractors. A major challenge in developing smart contracts is to guarantee that they are correct and free of security weaknesses, since bugs in their implementation may result in significant financial losses. For example, a single line that triggered integer overflow vulnerability in the BeautyChain (BECToken) contract caused a large number of stolen tokens - worth approximately $12 million. In the last few years, analysis and testing of smart contracts have raised considerable interest, and numerous techniques have been proposed to check the presence of vulnerabilities in them. However, the security and correctness of smart contracts is still a big question in academia and industry because of the problems in proposed techniques such as high false-positive rate, path explosion, and lack of support for Solidity constructs. This thesis presents novel approaches for automated test input generation and test effectiveness measurement for Solidity smart contracts. To achieve this, we address significant challenges related to smart contract execution, namely Solidity languagespecific data and features, and the extent of vulnerability detection. The thesis makes the following four contributions within automated testing. First, we propose an analysis, instrumentation, optimisation and code generation framework, SIF, using pre-defined helper functions that do not require any expert Solidity programming. This framework is capable of providing an interface to easily and effectively understand, manipulate and analyse Solidity code. Second, we present an automated framework, SolAnalyser, for detecting vulnerabilities over Solidity smart contracts. Our framework automatically inserts relevant assert statements as pre and post conditions for each arithmetic operation in the contract. Then, it generates a smart contract with relevant property assertions as output. Finally, our framework executes the generated smart contract with property assertions with automatically generated test inputs (from Contribution 3), and reports vulnerabilities, if any. Third, we implement different test input generation techniques for Solidity smart contracts. The techniques we implemented are based on fuzzing, genetic algorithm (GA), and satisfiability modulo theories (SMT) solver. Fuzzing relies on random input generation depending on contract interface. GA imitates the natural evolution to produce test input. SMT solver aims to solve given constraints for a practical subset of inputs. As the fourth contribution, we implement a framework, CovCal, to evaluate the effectiveness of different test input generation techniques. CovCal analyses them with respect to different coverage metrics, which are specific to Solidity structure, and fault-finding ability. We evaluate our frameworks, i.e., SIF, SolAnalyser and CovCal, and the testing techniques under different smart contract datasets that contain 1755 smart contracts and a wide variety of the Solidity constructs. We can infer the following from these experiments: (1) SIF is capable of supporting all the Solidity constructs, correctly instrumenting the Solidity code, and is user-friendly. (2) SolAnalyser is capable of detecting defined vulnerability types. (3) CovCal is capable of measuring the effectiveness of the test inputs automatically, in terms of code coverage and fault-finding capability in the Solidity smart contracts. (4) All of these techniques are able to generate valid test inputs for the Solidity smart contracts and detect any vulnerabilities in the tested contracts. All of these implementations and evaluations are open-sourced and have been used by other research groups