Logistic regression frequently outperformed propensity score methods, especially for large datasets: a simulation study

OBJECTIVE: In observational studies, researchers must select a method to control for confounding. Options include propensity score methods and regression. It remains unclear how dataset characteristics (size, overlap in propensity scores, exposure prevalence) influence the relative performance of the methods.

STUDY DESIGN: A simulation study to evaluate the role of dataset characteristics on the performance of propensity score methods, compared to logistic regression, for estimating a marginal odds ratio was conducted. Dataset size, overlap in propensity scores, and exposure prevalence were varied.

RESULTS: Regression showed poor coverage for small sample sizes, but with large sample sizes was relatively robust to imbalance in propensity scores and low exposure prevalence. Propensity score methods displayed suboptimal coverage as overlap in propensity scores decreased, which was exacerbated at larger sample sizes. Power of matching methods was particularly affected by lack of overlap, low exposure prevalence, and small sample size. The advantage of regression for large data size was reduced in sensitivity analysis with a complementary log-log outcome generation mechanism and unmeasured confounding, with superior bias and error but inferior coverage to matching methods.

CONCLUSION: Dataset characteristics influence performance of methods for confounder adjustment. In many scenarios, regression may be the preferable option.