Upper body detection and tracking in extended signing sequences

Journal article

The goal of this work is to detect and track the articulated pose of a human in signing videos of more than one hour in length. In particular we wish to accurately localise hands and arms, despite fast motion and a cluttered and changing background.

We cast the problem as inference in a generative model of the image, and propose a complete model which accounts for self-occlusion of the arms. Under this model, limb detection is expensive due to the very large number of possible configurations each part can assume. We make the following contributions to reduce this cost: (i) efficient sampling from a pictorial structure proposal distribution to obtain reasonable configurations; (ii) identifying a large number of frames where configurations can be correctly inferred, and exploiting temporal tracking elsewhere.

Results are reported for signing footage with challenging image conditions and for different signers. We show that the method is able to identify the true arm and hand locations with high reliability. The results exceed the state-of-the-art for the length and stability of continuous limb tracking.

Authors: Buehler, P; Everingham, M; Huttenlocher, D; Zisserman, A

• Publication status: Published
• Peer review status: Peer reviewed
• Publisher: Springer
• Journal: International Journal of Computer Vision
• Volume: 95
• Issue: 2
• Pages: 180-197
• Publication date: 2011-01-01
• DOI: 10.1007/s11263-011-0480-9
• EISSN: 1573-1405
• ISSN: 0920-5691
• Language: English
• Keywords: sign language; person tracking; gesture recognition; human pose estimation
• Subjects: Engineering & allied sciences