Performance evaluation of two published closed-loop control systems using bispectral index monitoring: A simulation study

Background. Although automated closed-loop control systems may improve quality of care, their safety must be proved under extreme control conditions. This study describes a simulation methodology to test automated controllers and its application in a comparison of two published controllers for Bispectral Index (BIS)-guided propofol administration. Methods: A patient simulator was developed to compare controllers. Using input scripts to dictate patient characteristics, target BIS values, and the time course of surgical events, the simulator continuously monitors the infusion pump under control and generates BIS values as a composite of modeled response to drug, perceived stimulation, and random noise. The simulator formats the output stream of BIS data as input to the controller under test to emulate the serial output of the actual BIS monitor. A published model-based controller and a classic proportional integral derivative controller were compared when using the BIS value as a controlled variable. Each controller was tested using a set of 10 virtual patients undergoing a fixed surgical profile that was repeated with BIS targets set at 30, 50, and 70. Controller performance was assessed using median (absolute) prediction error, divergence, wobble, and percentage time within BIS target range metrics. Results: The median prediction error was significantly smaller for the proportional integral derivative controller than for the model-based controller. The median absolute prediction error was smaller for the model-based controller than for the proportional integral derivative controller for each BIS target, reaching statistical significance for targets 30 and 50. Conclusions: When simulating closed-loop control of BIS using propofol, the use of a patient-individualized, model-based adaptive closed-loop system with effect site control resulted in better control of BIS compared with a standard proportional integral derivative controller with plasma site control. Even under extreme conditions, the modeled-based controller exhibited no behavioral problems