AGENT-BASED MODELLING: EXPLORING THE UNCERTAINTY AND SENSITIVITY OF EPIDEMIOLOGICAL MODELS

Abstract

The use of agent-based modelling for simulating the spread of the disease has seen increased use in recent years due to its ability to model scenarios that are infeasible or dangerous to recreate in the real world. The ability to execute a scenario with varying parameters, environments and populations is a valuable tool for the prediction of future disease-spread scenarios or to improve the understanding of previous events. The results of these models can aid in public health policy such as with the UK Covid-19 model which influenced the use of lockdowns within the country. Though the use of Agent-based modelling has many benefits there are some problems limiting its effectiveness. Agent-based models are complex and often difficult to understand even with sufficient documentation. The inclusion of uncertainty both aleatoric and epistemic within models can cause the results to vary each execution which makes the evaluation of any model outcomes difficult. Furthermore, The implementation of an agent-based model is complex and producing a model that is appropriately sensitive to aspects such as the input parameters used is challenging. Without sufficient evaluation, there is no way of knowing if a model is suitable for purpose and with epidemiological models being developed to inform health policy it is critical that the model is suitable or its influence could cause negative effects that are catastrophic to public health. In this thesis, we focus on two model evaluation challenges. The first is model uncertainty. Model uncertainty is the variation within the outputs of a model that is caused by either the stochastic processes within it with aleatoric uncertainty or the assumption of values with epistemic uncertainty. Uncertainty can affect the outputs of the model significantly thus if the uncertainty is not understood or sufficiently mitigated the results are less trustworthy as we cannot be sure if they are reflective of the model implementation or of random chance. The second area of evaluation is model sensitivity. The sensitivity of a model is a measure of how the input parameter values affect the outputs that are produced. Though we might expect a change in the parameter values to affect the outputs we also expect them to influence the outputs in ways that the model author intends. When effects that are not expected are encountered it may indicate issues within the model or the values chosen for the model parameters. Overall the contributions of this research can be summarised in four points: 1. A comprehensive analysis of 208 existing epidemiological agent-based modelling publications in which we evaluate the models described within them and 13 determine which methods for both sample size estimation and sensitivity analysis are prevalent. 2. We evaluate three methods for sample size estimation to determine which method is the most efficient for determining the number of executions that is suitable for a model. We also determine the consequences of not applying sample size estimation before performing an experiment by evaluating the author-defined sample sizes in which the majority were chosen without evaluation. 3. We evaluate the use of local and global sensitivity analysis to determine if the analysis of individual parameter influence is sufficient within epidemiological agent-based modelling or if a more comprehensive global analysis is required to fully evaluate the sensitivity of a model. 4. We evaluate a corpus of 21 epidemiological agent-based models providing an overview of them and also applying both the sample size estimation and sensitivity analysis methods that we evaluated in previous chapters. We then conclude on the effects of uncertainty and sensitivity within current models.

Date of Award	1 Aug 2024
Original language	English
Awarding Institution	The University of Manchester
Supervisor	Uli Sattler (Supervisor) & Bijan Parsia (Supervisor)