Real-time 3-D human body tracking using learnt models of behaviour

In this paper, we introduce a 3-D human-body tracker capable of handling fast and complex motions in real-time. We build upon the Monte-Carlo Bayesian framework, and propose novel prediction and evaluation methods improving the robustness and efficiency of the tracker. The parameter space, augmented with first order derivatives, is automatically partitioned into Gaussian clusters each representing an elementary motion: hypothesis propagation inside each cluster is therefore accurate and efficient. The transitions between clusters use the predictions of a variable length Markov model which can explain high-level behaviours over a long history. Using Monte-Carlo methods, evaluation of model candidates is critical for both speed and robustness. We present a new evaluation scheme based on hierarchical 3-D reconstruction and blob-fitting, where appearance models and image evidences are represented by mixtures of Gaussian blobs. Our tracker is also capable of automatic-initialisation and self-recovery. We demonstrate the application of our tracker to long video sequences exhibiting rapid and diverse movements. © 2007 Elsevier Inc. All rights reserved.