A Comparison of Machine Learning-Based Text Classifiers for Mapping Source Code to Architectural Modules

A mapping between a system's implementation and its software architecture is mandatory in many architecture consistency checking techniques. Creating such a mapping manually is a non-trivial task for most complex software systems. Machine learning-based text classification may be an highly effective tool for automating this task. How to make use of this tool most effectively has not been thoroughly investigated yet. This article presents a comparative analysis of three classifiers applied to map the implementations of five open-source systems to their architectures. The performance of the classifiers is evaluated for different extraction and preprocessing settings as well as different training set sizes. The results suggest that Logical Regression and Support Vector Machines both outperform Naive Bayes unless information about coarse-grained implementation structures cannot be exploited. Moreover, initial manual mappings of more than 15% of all source code files, or 10 files per module, do not seem to lead to a significantly better classification