Impact of Near Real-Time Warning Index in Simulated Clinical Assessment of Single Ventricle Infants
Presented By:
Fuchiang (Rich) Tsui, PhD; Victor Ruiz, PhD; Sachin Grover, MS; Lingyun Shi, MS; Allan Simpao, MD; Michael Goldsmith, MD
Children's Hospital of Philadelphia
goldsmitm1@chop.eduOverview:
Introduction: The Intensive Care Warning Index (I-WIN) is a near real-time clinical decision support (CDS) algorithm which assesses risk of deterioration in infants with single ventricle heart disease (Ruiz, JTCVS 2022). I-WIN identifies patients at risk of cardiac arrest, ECMO, or reintubation with AUC of 0.92 up to 4 hours prior to event. I-Win is one of many CDS tools that exist or are in development, however there is a paucity of data suggesting clinical impact. We sought to better understand how I-WIN CDS tool impacts a simulated clinician assessment.
Methods: In this simulation study, CICU clinicians reviewed 12 time-epochs of patients with single ventricle congenital heart disease during interstage period. Half of patients reviewed had a predefined deterioration event, matched with controls with no event. Each patient time epoch contained 12 hours of standard vital sign and laboratory data displayed in standard EMR views (Epic Inc, Verona, WI). Half of time epochs randomized to include I-WIN risk assessment (Intervention). Remaining half of time epochs had EMR Only. Clinicians assessed patient stability and risk of deterioration. Time to decision captured.
Results: N=13 clinical test users completed the study (n=5 fellows, n=6 nurse practitioners, n=1 physician assistant, n=1 hospitalist). 69% (N=9) had prior experience with clinical early warning systems. Cases with I-WIN clinical decision support (intervention) had a non-significant trend toward more accurate identification of patients with a future deterioration event (0.81 (SD 0.18), vs 0.68 (SD 0.17), p = 0.1. Non-significant trend toward improved sensitivity (0.74 vs 0.56, p = 0.15) in the intervention vs EMR only cases.
Conclusions: In a small cohort of test users not previously familiar or experienced with the I-WIN platform, there were non-significant trends toward improved accuracy of diagnosis and sensitivity when presented with a clinical decision support algorithm. Surprisingly, there was no difference in time to diagnosis between two groups. This simulation study does not re-create the time or work-pressure of the real ICU environment, future studies may pursue these pressures.
Future Steps: Assess impact of Risk Algorithm on clinical assessment in clinicians while distracted, under time pressure, fatigued. Assess how Risk Factor Impact variables effect clinician perception of stability, instability, and therapeutic targets. Develop prospective studies which capture impact of risk algorithm on clinical decision making.
