Improving the Reliability of a Nonprobability Web Survey: Application to Measuring Gender Pay Gap

Nonprobability web surveys suffer from selection and coverage biases and are generally not representative of the target population. To carry out statistical modelling in a nonprobability web survey, we explore different methods of statistical adjustments to compensate for biases through the use of a probability-based reference sample. We also show that we need to account for these biases when imputing missing data. The methods statistical adjustments include propensity score weighting with post-stratification and a technique called sample matching. For the substantive study, we use a nonprobability online web-survey taken from the Wage Indicator (WI) programme (www.wageindicator.org) to estimate the gender pay gap (GPG) using the 2016 WI survey data from the Netherlands. We use the 2016 EU-SILC data as the probability-based reference sample. Based on the study of GPG, we show that using propensity score weighting with post-stratification outperforms sample matching with respect to compensating for biases and improves the outcome of the Blinder-Oaxaca decomposition model in terms of the degree of similarity relative to patterns found in representative probability samples in the Netherlands.