Automatically detecting pain in video through facial action units

Lucey, Patrick J., Cohn, Jeffrey, Matthews, Iain, Lucey, Simon, Sridharan, Sridha, Howlett, Jessica M., & Prkachin, Kenneth M. (2010) Automatically detecting pain in video through facial action units. IEEE Transactions on Systems, Man, and Cybernetics - Part B : Cybernetics, pp. 1-11.

View at publisher


In a clinical setting, pain is reported either through patient self-report or via an observer. Such measures are problematic as they are: 1) subjective, and 2) give no specific timing information. Coding pain as a series of facial action units (AUs) can avoid these issues as it can be used to gain an objective measure of pain on a frame-by-frame basis. Using video data from patients with shoulder injuries, in this paper, we describe an active appearance model (AAM)-based system that can automatically detect the frames in video in which a patient is in pain. This pain data set highlights the many challenges associated with spontaneous emotion detection, particularly that of expression and head movement due to the patient's reaction to pain. In this paper, we show that the AAM can deal with these movements and can achieve significant improvements in both the AU and pain detection performance compared to the current-state-of-the-art approaches which utilize similarity-normalized appearance features only.

Impact and interest:

69 citations in Scopus
58 citations in Web of Science®
Search Google Scholar™

Citation counts are sourced monthly from Scopus and Web of Science® citation databases.

These databases contain citations from different subsets of available publications and different time periods and thus the citation count from each is usually different. Some works are not in either database and no count is displayed. Scopus includes citations from articles published in 1996 onwards, and Web of Science® generally from 1980 onwards.

Citations counts from the Google Scholar™ indexing service can be viewed at the linked Google Scholar™ search.

ID Code: 40908
Item Type: Journal Article
Refereed: Yes
Keywords: Active Appearance Models (AAMs), Emotion, Facial Action Coding System (FACS), Facial Action Units (AUs), Pain, Support Vector Machines (SVMs)
DOI: 10.1109/TSMCB.2010.2082525
ISSN: 1083-4419
Subjects: Australian and New Zealand Standard Research Classification > INFORMATION AND COMPUTING SCIENCES (080000) > ARTIFICIAL INTELLIGENCE AND IMAGE PROCESSING (080100) > Computer Vision (080104)
Divisions: Past > QUT Faculties & Divisions > Faculty of Built Environment and Engineering
Past > Schools > School of Engineering Systems
Copyright Owner: Copyright 2010 IEEE
Copyright Statement: Personal use of this material is permitted. However, permission to reprint/republish this material for advertising or promotional purposes or for creating new collective works for resale or redistribution to servers or lists, or to reuse any copyrighted component of this work in other works must be obtained from the IEEE.
Deposited On: 23 Mar 2011 21:55
Last Modified: 29 Feb 2012 14:31

Export: EndNote | Dublin Core | BibTeX

Repository Staff Only: item control page