How does Context Affect the Distribution of Software Maintainability Metrics?

Abstract—Software metrics have a variety of uses, for ex-ample, in defect prediction, effort estimation, and performance benchmarking for an organization against peers and industry standards. All these applications may depend on the context, such as the application domain, the programming language, and the number of downloads. In this paper, we aim to understand if the distributions of commonly used metrics do, in fact, vary with six contextual factors: application domain, programming language, age, lifespan, the number of changes, and the number of downloads. For this preliminary study we select 320 nontrivial software systems from SourceForge. These software systems are randomly sampled from nine popular application domains of SourceForge. We calculate 39 metrics commonly used to assess software maintainability for each software system and use Kruskal Wallis test and Mann-Whitney U test to determine if there are significant differences between the distributions with regard to each of the six contextual factors. We use Cliff’s delta to measure the magnitude of the differences. We find that all six contextual factors affect the distribution of 20 metrics and the programming language affects 35 metrics. We also provide preliminary suggestions on how each contextual factor may affect the distribution of metric values. We expect our results to help software benchmarking and other software engineering methods that rely on these commonly used metrics to be tailored to a particular context. Index Terms—context; contextual factor; metrics; static met-rics; software maintainability; benchmark; sampling; mining software repositories; large scale. I