Some methodologists have challenged the usefulness of statistics by arguing that the ontology implied by their use is inconsistent with the complex ontology of critical realism. Other critics of statistics take a strong social constructivist approach to research methodology. One problem with these sceptics' arguments is that they confuse the method of analytical statistics with the methodology of empiricism. We disentangle the two, and present a constructive argument supporting the cautious use of analytical statistics. The first part of the paper argues the case for an interpretive approach to statistical findings. In the middle of the paper an exemplar is presented showing that multivariate regression results can offer non-intuitive findings, can support non-atomistic interpretations, and can help underpin retroductive explanatory arguments. In exploring the nature of the warranted arguments that can arise after doing analytical statistics, we stress that explanations are emergent and do rest upon the workings of the statistical techniques and practices. We argue against seeing statistical techniques as a 'black box'. Instead, arguments can be developed, with justification resting in part upon the statistical results, in an audience-specific context of argumentation. The data which underlie statistical methods are not factual; the data are more like ficts than facts. Our argument is therefore that warranted arguments can be and are developed by social scientists who may use analytical statistics alongside other methods of research. Details of the argument can be explored further but it is important to establish that the sceptics' arguments are too dismissive of multivariate statistical analysis. © The Executive Management Committee/Blackwell Publishing Ltd. 2005.