Accounting for unexpected test responses through examinees' and their teachers' explanations

Researchers have developed indices to identify persons whose test results 'misfit' and are considered statistically 'aberrant' or 'unexpected' and whose measures are consequently potentially invalid, drawing the test's validity into question. This study draws on interviews of pupils and their teachers, using a sample of 31 10-year-olds who were flagged as most 'aberrant' in a standardised mathematics test. The children's and their teachers' explanations were analysed and attributed: (i) to item-, person- (self/other) and classroom-levels; and ii) according to causal dimensions. Children's and teachers' explanations were mostly in agreement in relation to unexpected negative results and they included references to previously well-cited sources of construct-irrelevant variance (e.g. ineffective test-taking strategies, careless mistakes) as well as construct-relevant variance (e.g. misconceptions, weaknesses in particular topics). Findings of this exploratory study are discussed from a test validity and attribution theory perspective: we conclude that this approach offers grounds for multi-level explanations of person misfit and that this qualitative research approach to unexpected responses is worthy of more attention. © 2010 Taylor & Francis.