Supporting Exploratory Testing by Automated Navigation Using the Model of System Under Test

Exploratory Testing (ET) is software testing technique, which is applicable to software development projects, in which test basis (design documentation) is not available, or at least not complete and consistent enough to allow the creation of e cient test cases. The principle of this technique is simultaneous learning, creation of the test cases and testing of the explored system under test (SUT). The key factor for the e ciency of this technique a documentation of explored tester's path in the SUT and the exercised test cases. When this is not being documented properly, ET loses its potential e ciency. Documentation of the explored paths in the SUT also allows more accurate reports of found defects, together with the possibility of the more systematic creation of the test cases during the exploratory testing process, preventing potential duplicities in executed tests. Created test cases can be used later in the next testing phases (retesting of xed defects or regression testing, for instance). Currently, a large ratio of web applications is usually developed without any or su - cient underlying models. In the most of the cases, this is a consequence of required low development costs and usually short time-to-market in the competitive software development environment. Nevertheless, the model is still implicitly present in the SUT code and with proper techniques, it is possible to reconstruct it from the SUT. Using the browser extension and server application, the framework automatically records selected actions of the exploratory testers in the SUT. Based on this recorded data, a screen- ow based model of the SUT is incrementally created and automatically updated. Based on this model and tester's position in the SUT during the testing process, the framework dynamically creates navigational test cases guiding the testers in the SUT and supports its e cient exploration. The recorded model consists of parts representing SUT pages, forms, input elements of the pages, and action elements as submit buttons and links. Besides that, home page, error pages and transition between SUT pages are de ned. The model constructs are accompanied by technical information and meta{data to support the model construction and generation of navigational test cases. In the test leader role, the tester can also extend the model with additional meta{ data. For example, he can prioritize certain pages or selected actions as a result of a risk analysis of the SUT. Moreover, he can de ne equivalence classes determining the suitable test data for particular inputs (e.g. text boxes in the forms) for the later generation of the navigational test cases. The SUT model is described formally and de ned constructs are used to document the proposal and the experiments verifying the framework functionality and practical e ciency. During the dynamical generation of navigational test cases, several navigational strategies can be selected. These strategies are based on combinations of the inputs like the parts of the SUT previously explored by an individual tester or all testers in the team, priority of the particular pages marked by the test team leader, or complexity of explored page elements. Performed experiments show that Exploratory Testing aided by this automated support is less resource demanding than Exploratory Testing performed manually only. With this support, also larger parts of the SUT were explored by the experiment participants in the de ned time. Also, as defect injection experiments have shown, that the proposed automated support helped the testing teams to detect more inserted software defects than ET performed as a manual process only. In particular, the main contributions of the dissertation thesis are as follows: Design and experimental implementation of automated method, which makes Exploratory Testing process more e cient in terms of spent resources, extent of explored SUT and found defects. Innovative combination of Exploratory Testing, Model{Based Testing and Reverse{ Engineering. De nition of formal model of the underlying system under test, which serves as a basis for the framework functionality. Design of initial navigational strategies, ranking functions, and test data strategies, used in the framework. These strategies are supporting the individual as well as team exploratory testing. Practical applicability of the proposed framework to industrial software development and testing projects.Katedra počítač