The Signal in the Noise: robust detection of performance 'outliers' in health services

Research output: Contribution to journalArticlepeer-review


To make the increasing amounts of data about the performance of public sector organisations digestible by decision makers, composite indicators are commonly constructed, from which a natural step is rankings and league tables. However, how much credence should be given to the results of such approaches? Studying English NHS maternity services (N=130 hospital trusts), we assembled and used a set of 38 indicators grouped into four baskets of aspects of service delivery. In the absence of opinion on how the indicators should be aggregated, we focus on the uncertainty this brings to the composite results.
We use a large two-stage Monte Carlo simulation to generate possible aggregation weights and examine the discrimination in the composite results. We find that positive and negative ‘outliers’ can be identified robustly, of particular value to decision makers for investigation for learning or intervention, however results in between should be treated with great caution.
Original languageEnglish
Pages (from-to)1102-1114
JournalJournal of the Operational Research Society
Issue number7
Early online date18 Oct 2018
Publication statusPublished - 15 May 2019


  • Health service
  • Hospitals
  • Management
  • Simulation
  • Quality


Dive into the research topics of 'The Signal in the Noise: robust detection of performance 'outliers' in health services'. Together they form a unique fingerprint.

Cite this