An algorithm for tuning an active appearance model to new data

Active Appearance Models [5] are widely used to match statistical models of shape and appearance to new images rapidly. They work by finding model parameters which minimise the sum of squares of residual differences between model and target image. Their efficiency is achieved by pre-computing the Jacobian describing how the residuals are expected to change as the parameters vary. This leads to a method of predicting the position of the minima based on a single measurement of the residuals (though in practise the algorithm is iterated to refine the estimate). However, the estimate of the Jacobian from the training set will only be an approximation for any given target image, and may be a poor one if the target image is significantly different from the training images. This paper describes a simple method of updating a representation of the Jacobian as the search progresses. This allows us to tune the AAM to the current example. Though useful for matching to a single image, it is particularly powerful when tracking objects through sequences, as it gives a method of tuning the AAM as the search progresses. We demonstrate the power of the technique on a variety of datasets.