Inter-rater reliability of explicit indicators of prescribing appropriateness

Mary P. Tully, Judith A. Cantrill

    Research output: Contribution to journalArticlepeer-review


    Objective: To assess the inter-rater reliability of 14 explicit indicators of appropriate long-term prescribing. Method: All available data required for the assessment of 59 long-term prescriptions started during a hospital admission for 25 patients were transcribed from the patients' medical records. These transcripts were presented in a standardised format and random order to four raters (two doctors and two pharmacists) who used the indicators to judge the appropriateness of each prescription. Debriefing interviews were held with each rater. An a priori level of acceptable agreement between the raters was set at a weighed kappa of 0.70. Results: There was no apparent difference between pharmacists and doctors for all findings, so data were combined. Two indicators showed poor agreement, three showed moderate agreement, and nine showed substantial or near perfect agreement, exceeding a weighted kappa of 0.70. There was excellent positive agreement as to which prescriptions were judged appropriate by the indicators, but much worse negative agreement as to which prescriptions were judged to be inappropriate. In the interviews, the raters remarked on the difficulty of applying explicit indicators when they routinely made implicit judgements about data in the medical records. Conclusion: Nine of the indicators achieved the required level of reliability and the negative agreement levels showed that this was the area that required greater improvement in future developments of the indicators. Further work needs to be conducted to investigate ways of the improving instructions on how to make explicit judgements and reducing the need for implicit or subjective assessments. © Springer 2005.
    Original languageEnglish
    Pages (from-to)311-315
    Number of pages4
    JournalPharmacy World and Science
    Issue number4
    Publication statusPublished - Aug 2005


    • Appropriate prescribing
    • Hospital
    • In-patient prescribing
    • Indicators
    • Prescribing indicators
    • Reliability
    • United Kingdom


    Dive into the research topics of 'Inter-rater reliability of explicit indicators of prescribing appropriateness'. Together they form a unique fingerprint.

    Cite this