Challenges in Markov chain Monte Carlo for Bayesian neural networks

Theodore Papamarkou, Jacob Hinkle, M. Todd Young, David Womble

Research output: Contribution to journalArticlepeer-review

175 Downloads (Pure)

Abstract

Markov chain Monte Carlo (MCMC) methods have not been broadly adopted in Bayesian neural networks (BNNs). This paper initially reviews the main challenges in sampling from the parameter posterior of a neural network via MCMC. Such challenges culminate to lack of convergence to the parameter posterior. Nevertheless, this paper shows that a non-converged Markov chain, generated via MCMC sampling from the parameter space of a neural network, can yield via Bayesian marginalization a valuable posterior predictive distribution of the output of the neural network. Classification examples based on multilayer perceptrons showcase highly accurate posterior predictive distributions. The postulate of limited scope for MCMC developments in BNNs is partially valid; an asymptotically exact parameter posterior seems less plausible, yet an accurate posterior predictive distribution is a tenable research avenue.
Original languageEnglish
Article number3
Pages (from-to)425-442
Number of pages18
JournalStatistical Science
Volume37
Issue number3
DOIs
Publication statusPublished - Aug 2022

Keywords

  • Bayesian inference
  • Bayesian neural networks
  • Markov chain Monte Carlo
  • convergence diagnostics
  • posterior predictive distribution

Fingerprint

Dive into the research topics of 'Challenges in Markov chain Monte Carlo for Bayesian neural networks'. Together they form a unique fingerprint.

Cite this