Calibration of uncertainty in the active learning of machine learning force fields

FFLUX is a Machine Learning Force Field that uses the Maximum Expected Prediction Error (MEPE) active learning algorithm to improve the efficiency of model training. MEPE uses the predictive uncertainty of a Gaussian Process to balance exploration and exploitation when selecting the next training sample. However, the predictive uncertainty of a Gaussian Process is unlikely to be accurate or precise immediately after training. We hypothesize that calibrating the
uncertainty quantification within MEPE will improve active learning performance. We develop and test two methods to improve uncertainty estimates: post-hoc calibration of predictive uncertainty using the CRUDE algorithm, and replacing the Gaussian Process with a Student-t Process. We investigate the impact of these methods on MEPE for single sample and batch sample active learning. Our findings suggest that post-hoc calibration does not improve the performance of active learning using the MEPE method. However, we do find that the Student-t Process can out perform active learning strategies and random sampling using a Gaussian Process if the training set is sufficiently large.