The measurement and improvement of maternity service performance through inspection and rating: An observational study of maternity services in acute hospitals in England

Objectives: To determine whether the prior performance of maternity services, as measured by Royal College of Obstetricians and Gynaecologists performance indicators, is associated with ratings by the Care Quality Commission at subsequent inspection, and whether performance changes occur after inspection.
Methods: We used hospital activity data from 176 maternity sites inspected between October 2013 and March 2016 to generate a set of performance indicators developed by the Royal College of Obstetricians and Gynaecologists. We linked these data to Care Quality Commission data on inspection dates and rating scores and used regression models, controlling for site level effects, to estimate the relationships between inspection ratings and performance indicators before and after inspections.
Results: Coefficients measuring the relationship between indicator performance and subsequent inspection rating score had wide confidence intervals which crossed zero suggesting no statistically significant relationship prior to inspection. The same absence of statistical significance was observed for changes in indicator performance after inspection.
Conclusions: The use of routine data for performance monitoring is becoming increasingly important as regular inspection is costly and regulators require accurate and timely intelligence. However, we found no statistically significant relationships between inspection ratings and performance indicators before or after inspections in maternity services. This calls into question the validity and reliability of the performance indicators, the inspection process and ratings, or both, as measures of performance.