Physics-Inspired Mathematical Reasoning with Transformers

Abstract

This research revolves around the exploration of the physics-related mathematical capabilities of transformer-based language models in tasks related to the classification and generation of mathematical language statements, with a particular focus on derivations of equations. We bolster models' ability to reason in a variety of physics-inspired settings, introduce and employ evaluation frameworks for exposing out-of-distribution generalisation failures of canonical models, present data generation methods to enhance the fine-grained mathematical inference of models and facilitate further experiments, explore derivation-centric improvements to common text generation metrics used to evaluate generative models, and introduce manually curated corpora (up to the level of physics research) that allows synthetically fine-tuned models to be evaluated on representations of real-world physics reasoning, and fuels an exploration into the perils of reproducing theoretical research in computer algebra systems. Collectively, this results in a thorough examination of the proficiency of contemporary models in emulating certain aspects of physics inference, precluded by a self-contained description of the Transformer, the mechanisms that allow transformer-based models to solve mathematical tasks, and the evolutionary path traced by research efforts as the field of artificial intelligence matured. We conclude with a discussion describing how hybrid approaches that employ language models to outsource mathematical inference to symbolic engines may circumvent models' tendency to fatally hallucinate incoherent argumentation, which indicates a paradigm shift towards neuro-symbolic methods.

Date of Award	1 Aug 2024
Original language	English
Awarding Institution	The University of Manchester
Supervisor	Andre Freitas (Supervisor) & Jonathan Shapiro (Supervisor)