A detailed model of the shape of anatomical structures can significantly improve the ability to segment such structures from medical images. Statistical models representing the variation of shape and appearance can be constructed from suitably annotated training sets. Such models can be used to synthesize images of anatomy, and to search new images to accurately locate the structures of interest, even in the presence of noise and clutter. In this paper we summarize recent work on constructing and using such models, and demonstrate their application to several domains. © 2004 The British Institute of Radiology.