Automatic randomized test generation strategy for aspect-oriented software

Software systems, nowadays, are used across the world to facilitate our daily life in different ways. Testing and verification of these software systems are very crucial with respect to safety, acceptance, and sidestepping failure damages. Aspect-oriented programming (AOP), as one of the prominent modularization techniques is gaining popularity as a methodology for contemporary software systems development. To realize the benefits of aspect-oriented programming and to produce validated and high quality aspect-oriented (AO) software, the programs developed by this programming paradigm should be effectively tested with attention to all of its characteristics, i.e. extension of new constructs and programming means for separation of concerns. The reason to emphasize is that, the AOP-specific defects stemmed from the unique characteristics of AOP can affect the quality of these programs in unusual ways and consequently diminish their general benefits, i.e. enhanced modularity and maintainability. In addition, these unique characteristics have made testing of AOP more sophisticated and challenging, though testing programs with simple functions is already hard and costly. To reduce the human effort and costs in assuring the correctness and general quality of aspect-oriented programs, it is desirable to automate testing of aspectoriented programs. In fact, automated testing has been a significant area of interest in current software testing research and has evolved greatly in recent years. However, it has to be said that there is comparatively little work on testing of AOP in the literature and very few and limited studies on automated testing of AOP. From a technical point of view, the existing studies on automated testing of AOP are not adequately effective in their underlying strategies used as the test generation has less AOP-specific focus and it is more founded on the knowledge of base code. In addition, lack of practical tools, lack of reliability and statistical evaluation from testing results with regard to the approach, difficulty in implementation of the strategy, and providing no automated test code generation are major issues associated with the existing studies. This obviously indicates an insufficiency of automated testing approaches, particularly automatic test generation, for the aspect-oriented programs at the current time and provides a primary motivation for leveraging the current techniques and/or developing new techniques for testing these programs. It is conjectured that applying recent techniques on testing and verification to AO-based systems is crucially important for making aspect-oriented software development (AOSD) success. As in the sequel, there is much advancement on testing and verification, like automated test generation using random-based and search-based techniques, coverage analysis, fault models, mutation analysis, and so forth. Many of those techniques are not yet applied or at least not properly investigated for AOP. Some would be trivial, but some would need some efforts to make them possible and leverage to testing of AOP. Random testing (RT) is an active and important research in software testing,which is a niche in practical settings due to its merits, e.g. fault-detection capacities at low cost, ease of implementation, reliability estimation of the software using the test results, and most importantly facility for automation. It can be said that RT is one of the most intuitive techniques for automatic test generation that has been widely studied and applied to test various programsfor decades and has formed the core parts of many useful testing approaches. On this regard, the idea behind random testing can be worthwhile and attractive to offer much promise in regard to AOP test automation problems since current research on testing of AOP, especially automated has not been adequately performed and is still in its infancy. So far, there is no mature testing approach in the context of AOP, considering the random testing techniques for automatic test generation of aspect-oriented programs, which can make this study as first in this sense. Therefore, to apply the random testing concept on AOP and to improve the degree of automation and the maturity of the automated AOP testing, in this thesis a rigorous automated test generation strategy, called RAMBUTANS, and its tool support based on the AspectJ programs were proposed. Furthermore, a thorough empirical study of nine AspectJ benchmark programs by means of mutation analysis was performed to compare RAMBUTANS and the four existing tomated AOP testing approaches (namely Wrasp, Aspectra, Raspect, and EAT) for testing aspects in terms of fault detection effectiveness and test effort efficiency. The results of experiment and statistical tests (p