Bayesian Deep Learning with Physics-informed Gaussian Processes

Abstract

The main focus of this thesis is a novel approach to Bayesian deep learning which incorporates physics-informed inductive biases into deep Gaussian processes. This framework allows for uncertainty-aware modeling of nonlinear dynamical systems in scenarios where we have prior knowledge regarding the form of the dynamics we expect a real-world system to exhibit, but the system is too complex to model mechanistically. We present two formulations of this model, one based upon a composition of weight-space GPs, and another which leverages variational inducing points and pathwise conditioning. We provide empirical evidence which shows the effectiveness of both formulations at extrapolating highly nonlinear dynamics and the behaviour of compositional systems. Additionally, we present a model designed to operate in scenarios where we have no prior knowledge regarding our data generating process. This method involves the analytical convolution of a series of filter processes with a shared latent process. This allows us to learn expressive Gaussian process covariances directly from data in situations where both input and output data is multidimensional, a setting in which it is challenging to reason about a suitable form for the covariance a priori. Experimental results are provided which show that our shallow model has the capacity to outperform hierarchical models, and learn covariances which deviate considerably from the simple covariances used in the latent and filter processes.

Date of Award	31 Dec 2024
Original language	English
Awarding Institution	The University of Manchester
Supervisor	Magnus Rattray (Supervisor) & Mauricio Alvarez (Supervisor)