Towards a Discourse-Level Natural Language Processing Algorithm: Characterizing Tumor Existence, Change of Existence, and its Progression from Unstructured Radiology Reports

Cancer has been the second leading cause of death in the US[1]. To provide care for cancer patients and retrospectively study this disease, clinicians and researchers need to manually analyze patient-level medical history to determine whether a tumor exists, has the state of existence changed, and does the change implicate disease progression. With the growing adoption of the electronic health records (EHRs), it is now possible to access these data and automate the discourse-level analysis on unstructured clinical texts using natural language processing (NLP) techniques.This thesis focuses on developing, training, and evaluating a transformer-based text classification algorithm that will capture contexts from unstructured radiology reports and output the discourse-level analysis on the tumor status and its progression through three conceptual frames: existence of a tumor, change of existence, and significance of change. This is the first clinical NLP work that conceptualize these representations using a wide range of systemic inferences, including contexts from presuppositions. The model shows promising results and can be extended to improve on casual reasoning, logical reasoning, numerical reasoning, and temporal reasoning in the future