Semi-automated Construction of Decision Rules to Predict Morbidities from Clinical Texts

Szeged to the second i2b2 Challenge in Natural Language Processing for Clinical Data. The challenge focused on the development of automatic systems that analyzed clinical discharge summary texts and addressed the following question: “Who’s obese and what co-morbidities do they (definitely/most likely) have?”. Target diseases included obesity and its 15 most frequent comorbidities exhibited by patients, while the target labels corresponded to expert judgments based on textual evidence and intuition (separately). Design: The authors applied statistical methods to preselect the most common and confident terms and evaluated outlier documents by hand to discover infrequent spelling variants. The authors expected a system with dictionaries gathered semi-automatically to have a good performance with moderate development costs (the authors examined just a small proportion of the records manually). Measurements: Following the standard evaluation method of the second Workshop on challenges in Natural Language Processing for Clinical Data, the authors used both macro- and microaveraged F1 measure for evaluation. Results: The authors submission achieved a microaverage F1 score of 97.29 % for classification based on textual evidence (macroaverage F1 76.22%) and 96.42 % for intuitive judgments (macroaverage F1 67.27%). Conclusions: The results demonstrate the feasibility of the authors approach and show that even very simple systems with a shallow linguistic analysis can achieve remarkable accuracy scores for classifying clinical records on a limited set of concepts. J Am Med Inform Assoc. 2009;16:601–605. DOI 10.1197/jamia.M3097