Traceability Link Recovery between Requirements and Models using an Evolutionary Algorithm Guided by a Learning to Rank Algorithm: Train control and management case

[EN] Traceability Link Recovery (TLR) has been a topic of interest for many years within the software engineering community. In recent years, TLR has been attracting more attention, becoming the subject of both fundamental and applied research. However, there still exists a large gap between the actual needs of industry on one hand and the solutions published through academic research on the other. In this work, we propose a novel approach, named Evolutionary Learning to Rank for Traceability Link Recovery (TLR-ELtoR). TLR-ELtoR recovers traceability links between a requirement and a model through the combination of evolutionary computation and machine learning techniques, generating as a result a ranking of model fragments that can realize the requirement. TLR-ELtoR was evaluated in a real-world case study in the railway domain, comparing its outcomes with five TLR approaches (Information Retrieval, Linguistic Rule-based, Feedforward Neural Network, Recurrent Neural Network, and Learning to Rank). The results show that TLR-ELtoR achieved the best results for most performance indicators, providing a mean precision value of 59.91%, a recall value of 78.95%, a combined F-measure of 62.50%, and a MCC value of 0.64. The statistical analysis of the results assesses the magnitude of the improvement, and the discussion presents why TLR-ELtoR achieves better results than the baselines.This work has been developed with the financial support of the Spanish State Research Agency and the Generalitat Valenciana under the projects DataME TIN2016-80811-P, ALPS RT12018-096411-B-100, ACIF/2018/171 and PROMETEO/2018/176, and co-financed with ERDF.Marcén, AC.; Lapeña, R.; Pastor López, O.; Cetina, C. (2020). Traceability Link Recovery between Requirements and Models using an Evolutionary Algorithm Guided by a Learning to Rank Algorithm: Train control and management case. Journal of Systems and Software. 163:1-24. https://doi.org/10.1016/j.jss.2020.110519124163Abeles, P., 2017. Efficient Java Matrix Library. [Online; accessed 12-April-2017], http://ejml.org/.Antoniol, G., Canfora, G., Casazza, G., De Lucia, A., & Merlo, E. (2002). Recovering traceability links between code and documentation. IEEE Transactions on Software Engineering, 28(10), 970-983. doi:10.1109/tse.2002.1041053Antoniol, G., Cleland-Huang, J., Hayes, J. H., Vierhauser, M., 2017. Grand Challenges of Traceability: The Next Ten Years. arXiv:1710.03129.Arcuri, A., & Briand, L. (2012). A Hitchhiker’s guide to statistical tests for assessing randomized algorithms in software engineering. Software Testing, Verification and Reliability, 24(3), 219-250. doi:10.1002/stvr.1486Arcuri, A., & Fraser, G. (2013). Parameter tuning or default values? An empirical investigation in search-based software engineering. Empirical Software Engineering, 18(3), 594-623. doi:10.1007/s10664-013-9249-9Beleites, C., Neugebauer, U., Bocklitz, T., Krafft, C., & Popp, J. (2013). Sample size planning for classification models. Analytica Chimica Acta, 760, 25-33. doi:10.1016/j.aca.2012.11.007Bengio, Y., Simard, P., & Frasconi, P. (1994). Learning long-term dependencies with gradient descent is difficult. IEEE Transactions on Neural Networks, 5(2), 157-166. doi:10.1109/72.279181Cetina, C., Font, J., Arcega, L., & Pérez, F. (2017). Improving feature location in long-living model-based product families designed with sustainability goals. Journal of Systems and Software, 134, 261-278. doi:10.1016/j.jss.2017.09.022Chandrashekar, G., & Sahin, F. (2014). A survey on feature selection methods. Computers & Electrical Engineering, 40(1), 16-28. doi:10.1016/j.compeleceng.2013.11.024Dang, V., 2013. The Lemur Project - Wiki - RankLib. [Online; accessed April-2017], http://sourceforge.net/p/lemur/wiki/RankLib/.Davis, L., 1991. Handbook of Genetic Algorithms.Dyer, D., 2016. The Watchmaker Framework for Evolutionary Computation (Evolutionary/Genetic Algorithms for Java). [Online; accessed 7-April-2016], http://watchmaker.uncommons.org/.Falessi, D., Cantone, G., & Canfora, G. (2013). Empirical Principles and an Industrial Case Study in Retrieving Equivalent Requirements via Natural Language Processing Techniques. IEEE Transactions on Software Engineering, 39(1), 18-44. doi:10.1109/tse.2011.122Frakes, W. B., Baeza-Yates, R., 1992. Information Retrieval: Data Structures and Algorithms.García, S., Fernández, A., Luengo, J., & Herrera, F. (2010). Advanced nonparametric tests for multiple comparisons in the design of experiments in computational intelligence and data mining: Experimental analysis of power. Information Sciences, 180(10), 2044-2064. doi:10.1016/j.ins.2009.12.010Hirzel, A. H., Le Lay, G., Helfer, V., Randin, C., & Guisan, A. (2006). Evaluating the ability of habitat suitability models to predict species presences. Ecological Modelling, 199(2), 142-152. doi:10.1016/j.ecolmodel.2006.05.017Hochreiter, S., & Schmidhuber, J. (1997). Long Short-Term Memory. Neural Computation, 9(8), 1735-1780. doi:10.1162/neco.1997.9.8.1735Hornik, K., Stinchcombe, M., & White, H. (1989). Multilayer feedforward networks are universal approximators. Neural Networks, 2(5), 359-366. doi:10.1016/0893-6080(89)90020-8Joachims, T., 1999. Svmlight: Support vector machine. SVM-Light Support Vector Machine http://svmlight. joachims. org/, University of Dortmund 19 (4).Kıraç, M. F., Aktemur, B., & Sözer, H. (2018). VISOR: A fast image processing pipeline with scaling and translation invariance for test oracle automation of visual output systems. Journal of Systems and Software, 136, 266-277. doi:10.1016/j.jss.2017.06.023Landauer, T. K., Foltz, P. W., & Laham, D. (1998). An introduction to latent semantic analysis. Discourse Processes, 25(2-3), 259-284. doi:10.1080/01638539809545028Lapeña, R., Font, J., Pastor, Ó., & Cetina, C. (2017). Analyzing the impact of natural language processing over feature location in models. ACM SIGPLAN Notices, 52(12), 63-76. doi:10.1145/3170492.3136052Lucia, A. D., Fasano, F., Oliveto, R., & Tortora, G. (2007). Recovering traceability links in software artifact management systems using information retrieval methods. ACM Transactions on Software Engineering and Methodology, 16(4), 13. doi:10.1145/1276933.1276934Mao, J., Xu, W., Yang, Y., Wang, J., Huang, Z., Yuille, A., 2014. Deep captioning with multimodal recurrent neural networks (M-RNN). arXiv:1412.6632.Meziane, F., Athanasakis, N., & Ananiadou, S. (2007). Generating Natural Language specifications from UML class diagrams. Requirements Engineering, 13(1), 1-18. doi:10.1007/s00766-007-0054-0Parizi, R. M., Lee, S. P., & Dabbagh, M. (2014). Achievements and Challenges in State-of-the-Art Software Traceability Between Test and Code Artifacts. IEEE Transactions on Reliability, 63(4), 913-926. doi:10.1109/tr.2014.2338254Piper, J. (1992). Variability and bias in experimentally measured classifier error rates. Pattern Recognition Letters, 13(10), 685-692. doi:10.1016/0167-8655(92)90097-jPoshyvanyk, D., Gueheneuc, Y.-G., Marcus, A., Antoniol, G., & Rajlich, V. (2007). Feature Location Using Probabilistic Ranking of Methods Based on Execution Scenarios and Information Retrieval. IEEE Transactions on Software Engineering, 33(6), 420-432. doi:10.1109/tse.2007.1016Rempel, P., & Mader, P. (2017). Preventing Defects: The Impact of Requirements Traceability Completeness on Software Quality. IEEE Transactions on Software Engineering, 43(8), 777-797. doi:10.1109/tse.2016.2622264Rus, I., & Lindvall, M. (2002). Knowledge management in software engineering. IEEE Software, 19(3), 26-38. doi:10.1109/ms.2002.1003450Salton, G., McGill, M. J., 1986. Introduction to modern information retrieval.Shabtai, A., Moskovitch, R., Elovici, Y., & Glezer, C. (2009). Detection of malicious code by applying machine learning classifiers on static features: A state-of-the-art survey. Information Security Technical Report, 14(1), 16-29. doi:10.1016/j.istr.2009.03.003Song, Q., Jia, Z., Shepperd, M., Ying, S., & Liu, J. (2011). A General Software Defect-Proneness Prediction Framework. IEEE Transactions on Software Engineering, 37(3), 356-370. doi:10.1109/tse.2010.90SPANOUDAKIS, G., & ZISMAN, A. (2005). SOFTWARE TRACEABILITY: A ROADMAP. Handbook Of Software Engineering And Knowledge Engineering, 395-428. doi:10.1142/9789812775245_0014Spanoudakis, G., Zisman, A., Pérez-Miñana, E., & Krause, P. (2004). Rule-based generation of requirements traceability relations. Journal of Systems and Software, 72(2), 105-127. doi:10.1016/s0164-1212(03)00242-5Sundaram, S. K., Hayes, J. H., Dekhtyar, A., & Holbrook, E. A. (2010). Assessing traceability of software engineering artifacts. Requirements Engineering, 15(3), 313-335. doi:10.1007/s00766-009-0096-6The Stanford Natural Language Processing Group (2017). https://nlp.stanford.edu/software/tagger.shtml. [Online; accessed 18-May-2017].VANNIEL, T., MCVICAR, T., & DATT, B. (2005). On the relationship between training sample size and data dimensionality: Monte Carlo analysis of broadband multi-temporal classification. Remote Sensing of Environment, 98(4), 468-480. doi:10.1016/j.rse.2005.08.011Walczak, S., & Cerpa, N. (1999). Heuristic principles for the design of artificial neural networks. Information and Software Technology, 41(2), 107-117. doi:10.1016/s0950-5849(98)00116-5Jialei Wang, Peilin Zhao, Hoi, S. C. H., & Rong Jin. (2014). Online Feature Selection and Its Applications. IEEE Transactions on Knowledge and Data Engineering, 26(3), 698-710. doi:10.1109/tkde.2013.32Watkins, R., & Neal, M. (1994). Why and how of requirements tracing. IEEE Software, 11(4), 104-106. doi:10.1109/52.300100Winkler, S., & von Pilgrim, J. (2009). A survey of traceability in requirements engineering and model-driven development. Software & Systems Modeling, 9(4), 529-565. doi:10.1007/s10270-009-0145-0Zhang, Z., Chen, L., Tian, P., Su, J.,. Source localization in an ocean waveguide using supervised machine learning. Computing 11, 5.Zhou, Z.-H., Feng, J., 2017. Deep Forest: Towards an Alternative to Deep Neural Networks. arXiv:1702.08835