Managing mismatches in COTS-based development.

The prospect of reducing the time, cost and risk of system development has increased the interest in developing systems from COTS (Commercial-Off-The-Shelf) products. The development of complex COTS-based systems is known to be an intricate and risk prone process. There are three main reasons for this. Firstly, suppliers develop COTS products with the objective of satisfying the needs of the marketplace rather than the specific needs of the acquirer organization. Secondly, COTS products are often delivered as black-boxes, which means that the understanding of COTS features is frequently partial and uncertain on the part of the acquirer. Thirdly, in order to sustain competitive advantage, suppliers regularly modify their products, hence forcing customers to update their systems. These challenges do not occur in traditional software development, they are particular attributes of COTS-based development (CBD). As a result, new processes, methods and models are needed to support the development of COTS-based systems. The selection of COTS products is one of the most important activities taking place in the context of CBD. It involves the assessment of how well COTS products satisfy customer requirements. Due to the nature of COTS, mismatches may occur between what is wanted from the system (i.e. customer requirements) and what the system is able to provide (i.e. its features). In addition, a number of risks may arise from these mismatches, such as insufficient COTS adherence to requirements, low confidence in COTS quality, and unwanted COTS features. We argue that the successful selection of a suitable COTS product depends on the effective analysis of mismatches and management of risks. This thesis proposes a novel method, called TAOS (Tradeoff Analysis for COTS-based Systems), to guide the selection of COTS products. TAOS offers a systematic approach to assess the suitability of COTS products by exploring mismatches, handling risks and suggesting possible tradeoffs to be made. The method uses a goal-oriented approach to specify the requirements of the acquirer organization. We demonstrate how utility theory can be used to compare COTS alternatives by examining the degree to which COTS products satisfy requirements, and therefore inform the decision making process. As a way to complement the quantitative assessment obtained from the use of utility theory, we present a set of templates to build exploratory scenarios and define heuristics to facilitate the tradeoff analysis. We also present a strategy to identify and manage risks. To establish the effectiveness of TAOS to improve the quality of decisions made during the selection process, we have conducted a number of case studies in different domains