Empirical evidence that disease prevalence does affect diagnostic test performance – the effects of prevalence on the interpretation of x-rays by junior doctors

BH. Willis

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

Abstract

Background: Recently it has been recognised that the sensitivity and specificity may be affected by the prevalence of disease1.Objective: To evaluate the effects of the prevalence of abnormality on junior doctors’ performance in interpreting x-rays.Method: A systematic sample of 2593 patients’ records was collected from an attending cohort at a UK emergency department. In the sample 1053 x-rays were interpreted by junior doctors following their triage into high and low probability of abnormality populations by radiographers. Following exclusion of 86 x-rays due to incomplete data, 967 were analysed. The main outcomes were sensitivity, specificity, likelihood ratios, diagnostic odds ratios and ROC curve.Results: For the high probability (77%) and low probability (13%) x-rays, the overall sensitivities were 94.1 (89.5 – 96.8) and 75.0 (65.5 - 82.6) and the specificities were 56.0 (42.3 - 68.8) and 92.3 (90.0 - 94.1) respectively. These were statistically significant as were the differences in the positive likelihood ratio between the two populations: 2.14 (1.56 - 2.93) and 9.78 (7.32 - 13.07) respectively. To assess potential differences in patient spectrum between the high and low probability populations the distributions of x- ray type were compared. Although in some individual categories there were differences in the proportions of x-rays examined, logistic regression showed that overall, x-ray type did not significantly affect test performance. Differences in the diagnostic odds ratios were also found not to be significant which is consistent with a common ROC curve and with doctors changing their implicit threshold between the two populations.Conclusions: The results support the hypothesis that the sensitivity and specificity of a diagnostic test may be affected by the disease prevalence. More generally this has implications for clinicians when applying multiple tests, as each test leads to a revision of the probability of disease, and this may influence the performance of subsequent tests. Ultimately this may affect the transferability of study results to practice.[1] Leeflang MM, Bossuyt PM, Irwig L., Diagnostic test accuracy may vary with prevalence: implications for evidence-based diagnosis, J Clin Epidemiol. 2009 62(1):5-12
Original languageEnglish
Title of host publicationhost publication
Place of Publicationhttp://www.haps.bham.ac.uk/publichealth/diagnostic-research/conference/docs/abstract_book2010.pdf
PublisherUniversity of Birmingham
Publication statusPublished - 1 Jul 2010
EventMethods for evaluating medical tests and biomarkers, Second International Symposium - University of Birmingham, UK
Duration: 1 Jul 20102 Jul 2010

Conference

ConferenceMethods for evaluating medical tests and biomarkers, Second International Symposium
CityUniversity of Birmingham, UK
Period1/07/102/07/10

Fingerprint

Dive into the research topics of 'Empirical evidence that disease prevalence does affect diagnostic test performance – the effects of prevalence on the interpretation of x-rays by junior doctors'. Together they form a unique fingerprint.

Cite this