Analytic Error as an Important Component of Total Survey Error: Results from a Meta-Analysis

This chapter describes the results of a meta-analysis that sought to quantify the frequency with which researchers do not use correct analytic techniques for complex sample survey data when publishing secondary analyses of survey data arising from complex samples. It explores whether characteristics of the journals in which these articles were published were related to the prevalence of various errors. The chapter defines analytic error as the failure of the survey data user to employ appropriate estimation methods when analyzing the collected survey data. Survey organizations strive to minimize important sources of total survey error (TSE) paradigm, and these efforts come at significant expense to funding agencies and the tax-paying public in general. The chapter presents results from the analyses of these data, which suggest that analytic errors may in fact be quite prevalent in published analyses of complex sample survey data.