There is growing interest in the use of clinical prediction models (CPMs) to aid decision making across healthcare. CPMs provide risk estimates for the presence of disease, or future outcomes, given current information about a patient. The pipeline of getting a CPM into clinical practice involves, (i) model development where a dataset is used to estimate the model parameters, (ii) model validation, where the predictive performance of the model is evaluated, (iii) impact assessment, where the clinical impact of the CPM is evaluated, and then finally (iv) model implementations into practice. It is commonly the case that once a model has been implemented, the model coefficients/parameters remain fixed, or at best are updated at arbitrary time points. However, healthcare and patient populations experience changes in terms of processes and case-mix, respectively, which means the covariate-outcome associations of the CPM also need to change, which is not reflected in most CPMs to-date. This results in the accuracy of the CPMs diminishing over time. This is known as calibration drift and is one of the major pitfalls of CPMs to date. Dynamic prediction models are a possible solution as the model parameters are not fixed and they attempt to acknowledge/model the temporal nature of the data. This thesis explores the challenges of CPMs in the presence of calibration drift. The aims of the thesis are to (a) provide a comprehensive understanding of the methodology and challenges with dynamic modelling, (b) compare the predictive performance of the different models, and (c) to develop a method to address the problem of arbitrary updating. Chapter 2 identifies existing methods used for dynamic prediction modelling through a review of the literature and highlights the current methodological challenges in dynamic prediction modelling. Chapter 3 discusses potential solutions to overcome the challenges described in chapter 2, leading to the suggestion of dynamic prediction systems, a way to continuously update and monitor a model over time. Following on from these chapters, chapters 4 and 5 compare the methods identified in chapter 2 through a simulation study and real-world data examples in cardiovascular disease. Despite the issues identified in chapters 2 and 3, chapters 4 and 5 found dynamic models perform as well as or better than non-dynamic models, which are currently the norm in the field. Building on this, chapter 6 develops a solution to one of the major challenges in predictive modelling, continuous monitoring and feedback, and illustrates the novel approach through simulation. Generally, this thesis has the potential to improve performance and monitoring of prediction models, especially in presence of performance drift, by moving away from the current CPM framework and methods towards the proposed dynamic prediction systems. Practically, the thesis has used traditional and novel methodology to further the field of CPM development and validation.
- Clinical prediction model
- Model updating
- Dynamic models
DYNAMIC MODELLING METHODS FOR CLINICAL PREDICTION MODEL UPDATING AND MONITORING: IDENTIFICATION AND COMPARISON OF DYNAMIC MODELS FOR HEALTHCARE USE
Jenkins, D. (Author). 1 Aug 2022
Student thesis: Unknown