Combining probability and non-probability samples to reduce survey errors and survey costs: a proof-of-concept study

Carefully designed probability-based sample surveys, which form the backbone of social science and policy-relevant research, are under significant threat due to increasing costs and declining response rates. As a result, many survey organizations have abandoned probability sampling in favour of cheaper but less accurate non-probability sampling. In this project, we evaluate a new method of combining both sampling methods under a Bayesian inferential framework. The method is designed to utilize non-probability samples to boost the efficiency of survey estimates derived from probability samples of small sizes. The boost in efficiency may lead to potential cost savings to the survey organisation. However, the use of non-probability samples may lead to biased estimates. We will evaluate under which conditions gains in efficiency offset bias in estimates of voter turnout using simulated data and real data from ten probability and non-probability surveys conducted simultaneously in Germany.

In this project, we have investigated methods for reducing the cost of measuring public opinion and other characteristics of the society. We have demonstrated that this can be achieved by carrying out two types of surveys in the same time. The first type is a traditional survey, such as by telephone or face to face, which relies on several restrictive assumptions and provides balanced measurement but tends to be costly. The second type is a new form of a survey, such as information collected using Internet survey, has a higher risk of error in measurement but tends to be much cheaper than the traditional one and information can be collected from many more persons. When we use a small traditional survey supplemented by, say, survey carried out on the Internet, the final measurement will be as precise as a large and costly traditional survey alone but will be achieved at a fraction of its cost.