Improving Syndromic Data Quality through Implementation of Error Capture Module

Objective: To streamline emergency department data processing in Oregon ESSENCE (Oregon’s statewide syndromic surveillance) by systematically and efficiently addressing data quality issues among submitting hospital systems.Introduction: Oregon Public Health Division (OPHD), in collaboration with The Johns Hopkins University Applied Physics Laboratory, implemented Oregon ESSENCE in 2011. ESSENCE is an automated, electronic syndromic surveillance system that captures emergency department data from hospitals across Oregon. While each hospital system sends HL7 2.5.1-formatted messages, each uses a uniquely configured interface to capture, extract, and send data. Consequently, ESSENCE receives messages that vary greatly in content and structure. Emergency department data are ingested using the Rhapsody Integration Engine 6.2.1 (Orion Health, Auckland, NZ), which standardizes messages before entering ESSENCE. Mechanisms in the ingestion route (error-handling filters) identify messages that do not completely match accepted standards for submission. A sub-set of these previously-identified messages with errors are corrected within the route as they emerge.Existence of errors does not preclude a message’s insertion into ESSENCE. However, the quality and quantity of errors determine the quality of the data that ESSENCE uses. Unchecked, error accumulation also can cause strain to the integration engine. Despite ad-hoc processes to address errors, backlogs accrue. With no meta-data to assess the importance and source of backlogged errors, the ESSENCE team had no guide with which to mitigate errors. The ESSENCE team needed a way to determine which errors could be fixed by updating the Rhapsody Integration Engine and which required consultation with partner health systems and their data vendors. To formally address these issues, the ESSENCE team developed an error-capture module within Rhapsody to identify and quantify all errors identified in syndromic messages and to use as a guide to prioritize fixing new errors.Methods: Members of OPHD’s informatics team and the Oregon ESSENCE team met to brainstorm solutions to error accumulation and message-processing inefficiencies. The team agreed that existing infrastructure and resources were sufficient to accomplish this project. Using Rhapsody, the team created filters that generated error messages each time an HL7 message failed to validate pre-determined message parameters (a standard HL7 2.5.1 syndromic message definition). In order to capture information about errors that were currently being fixed by the ingestion route, two filters were inserted into the processing route: one before and one after previously-existing error-fixing message-modifiers. The team created a FileMaker database to collect information about each error identified, including submitter, location of error (segment, field, component), type of error (too long for field, not in value set, doesn’t adhere to correct message structure). The team enabled error-capture for 81 days (June 16 – September 5, 2017) at which point they evaluated error data so as to guide repair of message modifying filters within the integration engine.Results: The module captured 16,273,963 error messages over 81 days. The two error capture filters (before and after existing modifiers) each generated 50% of the error messages. The module identified errors across seven HL7 message segments (DG1, EVN, OBX, PID, PR1, PV1, PV2). One submitter produced 87% of the error messages. Of those, 93% were errors in two fields in the PID segment. Based on the results, ESSENCE team members contacted this submitter and resolved this error, greatly reducing the ingestion and error assessment burden of Rhapsody and the development team. Currently the ESSENCE team is still analyzing the rest of the errors, applying fixes and contacting submitting facilities as needed. Once completed, the ESSENCE team will enable the error capture module bi-annually to continue refinement of its system strategically guide data quality work.Conclusions: Oregon ESSENCE developed a tool to evaluate errors in emergency department HL7 messages it receives for syndromic surveillance. Its quick development and reusability make it a cost-effective and sustainable data quality solution for focusing effort in regional installations of syndromic surveillance