Rasch Analysis of the Antimicrobial Self-Assessment Toolkit for NHS Trusts (ASAT v17)

Objectives: The Antimicrobial Self-Assessment Toolkit for NHS Trusts (ASAT) was developed to evaluate hospital-based antimicrobial stewardship programmes. Iterative validity investigations of the ASAT were used to produce a 91-item ASAT v17 utilising qualitative methodology. Rasch analysis was used to generate question (item) behaviour estimates and to investigate the validity of ASAT v17.Methods: In 2012, the Partial Credit Model (PCM) was used to analyse ASAT responses from 33 NHS Trusts within England. WINSTEPS ® outputs such as fit statistics and respondent/item maps were examined to determine unidimensionality, item discrimination and item hierarchy. Ordinary Least Squares (OLS) regression modelling was used to determine the predictive validity using NHS Trust ability estimates generated from the PCM and corresponding Clostridium difficile rates.Results: Each domain contained items that were misfitting the PCM (with INFIT MNSQ <0.7 or >1.3), except Domain 3. Subsequent iterative item removal had a negligible effect on the fit indices within most ASAT domains. Scale analysis demonstrated that most items were productive for measurement (n=81). Respondent/item maps showed ceiling effects (n=3) and floor effects (n=1) within ASAT domains. OLS regression modelling identified that there was limited predictive validity due to the small positive correlation between the predictor and outcome variables for participating hospitals (ρ =0.146; p=0.418).Conclusions: Rasch analysis was an effective measurement technique for evaluating the validity of ASAT v17 by providing evidence that each sub-scale and the overall scale demonstrated unidimensionality (construct validity). Improved item targeting may be required to improve item discrimination within the toolkit.