The Usability of Crowdsourcing in Diagnostic Radiology Problem Solving

Abstract

This thesis explores crowdsourcing as a solution to the radiologist shortage, exacerbated by the COVID-19 pandemic and rising demand. It questions the reliance on expert radiologists for image interpretation. It introduces the Expertise-Stratified Crowdsourcing Model (ESCM), inspired by the Dreyfus framework, to enhance diagnostics through the crowd's diverse expertise. Adopting the Value-Based Healthcare (VBH) framework, the research investigates non-expert groups' potential to interpret knee MRI scans for meniscal tears, hypothesising that their varied expertise levels could impact diagnostic accuracy. The aim is to evaluate the impact of non-expert expertise on diagnostic accuracy, addressing the challenge of delivering high-value radiology services. A quantitative, quasi-experimental design was used to compare the diagnostic accuracy of non-expert groups with that of radiology experts. Orthopaedic arthroscopy reports served as the gold standard, and retrospective pre-operative MRI data from Israel (2012-2016) were utilised. The study was conducted in controlled labs for non-experts and standard clinical settings for experts. The study focused on the effect of different expertise levels on meniscal tear diagnoses from MRI scans, both before and after targeted training. Participants included three non-expert groups - high-school graduates, physiotherapy students, and MRI radiographers (12 members each) - and an expert group of three MRI musculoskeletal imaging specialists. They interpreted 32 knee MRI cases, processing each case to diagnose medial and lateral meniscus before and after training, yielding 1,536 assessments per non-expert group. Metrics included Percentage of Agreement, Sensitivity, Specificity, Positive Predictive Value (PPV), Negative Predictive Value (NPV), and kappa coefficients, with pre-and post-training comparisons using Pearson correlation coefficients. Results showed significant improvements in non-experts' diagnostic accuracy post-training, with some groups surpassing expert radiologists. MRI Radiographers achieved a kappa of 0.56 and Agreement of 78.13%, surpassing the expert group's kappa of 0.34 and Agreement of 68.75%. Strong positive correlations in diagnostic accuracy measures affirm the impact of targeted training on non-expert diagnostic capabilities. This study advances understanding of crowdsourcing in diagnostic radiology, highlighting its potential for complex problem-solving tasks and proposing ESCM as a scalable solution for radiology interpretation.

Date of Award	1 Aug 2025
Original language	English
Awarding Institution	The University of Manchester
Supervisor	Dimitrija Kalanoski (Supervisor), Anita Greenhill (Supervisor) & Ruth McDonald (Supervisor)