Two Datasets of Defect Reports Labeled By a Crowd of Annotators of Unknown Reliability

Classifying software defects according to any defined taxonomy is not straightforward. In order to be used for automatizing the classification of software defects, two sets of defect reports were collected from public issue tracking systems from two different real domains. Due to the lack of a domain expert, the collected defects were categorized by a set of annotators of unknown reliability according to their impact from IBM's orthogonal defect classification taxonomy. Both datasets are prepared to solve the defect classification problem by means of techniques of the learning from crowds paradigm (Hernández-González et al. [1]). Two versions of both datasets are publicly shared. In the first version, the raw data is given: the text description of defects together with the category assigned by each annotator. In the second version, the text of each defect has been transformed to a descriptive vector using text-mining techniques.This work has been partially supported by the Basque Government(IT609-13,ElkartekBID3A), the Spanish Ministry of Economy and Competitiveness(TIN2016-78365-R) and the University-Society Project15/19(Basque Government and University of the Basque Country UPV/EHU).JoseA.Lozano is also supported by BERC Program 2014–2017(Basque Government) and Severo Ochoa Program SEV- 2013-0323 (Spanish Ministry of Economy and Competitiveness).Daniel Rodriguez carriedo utthis work while visiting Oxford Brookes University.He is partlys upported by projects Badge People TIN2016–76956-C3-3-R.Wewould like to thank Varsha Veerappa and the anony mousannotators for their help with data collection