Does the choice of risk-adjustment model influence the outcome of surgeon-specific mortality analysis? A retrospective analysis of 14 637 patients under 31 surgeons

Objectives: To compare implications of using the logistic EuroSCORE and a locally derived model when analysing individual surgeon mortality outcomes. Design: Retrospective analysis of prospectively collected data. Setting: All NHS hospitals undertaking adult cardiac surgery in northwest England. Patients: 14 637 consecutive patients, April 2002 to March 2005. Main outcome measures: We have compared the predictive ability of the logistic EuroSCORE (uncalibrated), the logistic EuroSCORE calibrated for contemporary performance and a locally derived logistic regression model. We have used each to create risk-adjusted individual surgeon mortality funnel plots to demonstrate high mortality outcomes. Results: There were 458 (3.1%) deaths. The expected mortality and receiver operating characteristic (ROC) curve values were: uncalibrated EuroSCORE -5.8% and 0.80, calibrated EuroSCORE -3.1% and 0.80, locally derived model -3.1% and 0.82. The uncalibrated EuroSCORE plot showed one surgeon to have mortality above the northwest average, and no surgeon above the 95% control limit (CL). The calibrated EuroSCORE plot and the local model showed little change in surgeon ranking, but significant differences in identifying high mortality outcomes. Two of three surgeons above the 95% CL using the calibrated EuroSCORE revert to acceptable outcomes when the local model is applied but the finding is critically dependent on the calibration coefficient. Conclusions: The uncalibrated EuroSCORE significantly overpredicted mortality and is not recommended. Instead, the EuroSCORE should be calibrated for contemporary performance. The differences demonstrated in defining high mortality outcomes when using a model built for purpose suggests that the choice of risk model is important when analysing surgeon mortality outcomes.