Regulatory conceptualisation and assessment of improvement capability

Background

Regulation continues to be a policy used to improve healthcare quality. However, regulation is criticised as a top down, ineffective and reactive way of improving quality. Partly in response to these criticisms, health care regulatory agencies are increasingly concerned not just with assessing the current performance of the organisations they regulate, but with assessing and influencing their underlying improvement capability and prospective performance trajectory. However, little is known about regulatory perspectives of improvement capability. This research study examines how improvement capability is conceptualised and assessed in practice by six organisational regulatory agencies in the UK.

Method

A comparative qualitative study was conducted focusing on hospital-based care, accounting for the majority of UK health care expenditure. Six UK agencies were selected and all participated. Data sources included 90 regulatory policy documents, 48 interviews from a cross-section of staff and 30 assessment reports (five per agency) from 2013–2015. Qualitative analysis of the data sources was conducted using an a priori framework of eight dimensions of improvement capability identified from the literature. The framework was inductively developed from a literature review of 70 instruments used for improvement capability assessment. The framework consists of the following eight dimensions of improvement capability: organisational culture; leadership commitment; employee commitment; service user focus; stakeholder and supplier focus; process improvement and learning; strategy and governance; and data and performance.

Outcome

The analysis revealed that the dimensions of process improvement and learning and strategy and governance were most frequently found. Other dimensions were found less frequently, with service-user focus being the least frequent, and this skewed pattern was consistent across agencies. Three themes emerged from the empirical data. First, it is problematic to define and operationalise improvement capability. Policy documents and interviews stressed the importance of developing improvement capability, but did not articulate consistently what was meant by improvement capability. Second, the assessment instrument relied on out-of-date and infrequently measured data. Third, there was variable understanding of improvement capability, causing variation and assessment bias through self-confessed knowledge gaps. Effects of changes.

Conclusion

It may be that dimensions which are relatively easy to ‘measure’ (such as strategy and governance documents) dominate assessment processes rather than less tangible dimensions. Alternatively, it may indicate gaps in regulatory agencies’ assessment instruments, knowledge of improvement capability, or practical difficulties in operationalising regulatory intentions to reliably assess improvement capability. The study highlights the need for regulatory agencies to further conceptualise improvement capability to inform assessment. This will strengthen agencies’ assessment, diagnosis and prediction of performance trajectories and support the development of tailored regulatory interventions.