Evaluating the development and validation of empirically-derived prognostic models for pressure ulcer risk assessment: a systematic review

Background

Researchers advocate developing empirically-derived prognostic models to predict pressure ulcer risk. However, there remains a scarcity of evidence about the performance and clinical value of these models.

Objectives

To identify and describe empirically-derived models for predicting pressure ulcer risk; to assess the predictive performance of these models; and to evaluate their clinical impact in reducing pressure ulcer incidence.

Methods

We performed a comprehensive database search up to February 2017 and searched other resources to identify longitudinal studies that developed and/or validated prognostic models for predicting pressure ulcer risk and studies evaluating the clinical effects of such models. There were no language or publication date restrictions. Two reviewers independently conducted study selection. Using a pre-prepared data extraction form, one reviewer collected data on the characteristics and performance of the included models and assessed study risk of bias. A second reviewer checked all the data. Using narrative synthesis, we summarised the characteristics of the included studies and models. Using meta-analysis, we combined performance (discrimination and calibration) measurement statistics for relevant models.

Results

We included 24 studies with 28 data sources in the review and identified 22 models that were developed using these data. Of the 22 models, only seven had further external validations (one model was validated twice). In development, a third of models used univariate analysis alone to identify statistically significant predictors for subsequent multivariable analysis; and nine of the 16 developed models were formed using stepwise selection processes in multivariable analysis. Missing data were often incompletely reported, and continuous predictors were correctly handled in only two models (e.g., using restricted cubic spline). Sample sizes of the model development studies were small with 13 models involving fewer than 10 events per variable. The risk of bias associated with the development of all 22 models and eight validations was judged as high or unclear. The predictive performance was reported as: c-statistic point estimates ranging from 0.65 to 0.89, and total Observed:Expected risk ratios between 0.94 and 1.00. Compared with heuristic tools, relevant included models had better discrimination and calibration. No eligible study was identified that evaluated the clinical impact of any included model.

Conclusions

Whilst many prognostic models for predicting ulcer risk have been developed few have been validated. The methods used for model development are generally flawed which reduces the potential for using these models in practice. Future research should address these weaknesses.