Using viewer’s facial expression and heart rate for sports video highlights detection

Chakraborty, Prithwi Raj, Zhang, Ligang, Tjondronegoro, Dian W., & Chandran, Vinod (2015) Using viewer’s facial expression and heart rate for sports video highlights detection. In ACM International Conference on Multimedia Retrieval (ICMR 2015), 23-26 June 2015, Shanghai, China. (Unpublished)



Viewer interests, evoked by video content, can potentially identify the highlights of the video. This paper explores the use of facial expressions (FE) and heart rate (HR) of viewers captured using camera and non-strapped sensor for identifying interesting video segments. The data from ten subjects with three videos showed that these signals are viewer dependent and not synchronized with the video contents. To address this issue, new algorithms are proposed to effectively combine FE and HR signals for identifying the time when viewer interest is potentially high. The results show that, compared with subjective annotation and match report highlights, ‘non-neutral’ FE and ‘relatively higher and faster’ HR is able to capture 60%-80% of goal, foul, and shot-on-goal soccer video events. FE is found to be more indicative than HR of viewer’s interests, but the fusion of these two modalities outperforms each of them.

Impact and interest:

1 citations in Scopus
Search Google Scholar™

Citation counts are sourced monthly from Scopus and Web of Science® citation databases.

These databases contain citations from different subsets of available publications and different time periods and thus the citation count from each is usually different. Some works are not in either database and no count is displayed. Scopus includes citations from articles published in 1996 onwards, and Web of Science® generally from 1980 onwards.

Citations counts from the Google Scholar™ indexing service can be viewed at the linked Google Scholar™ search.

Full-text downloads:

33 since deposited on 22 Mar 2015
15 in the past twelve months

Full-text downloads displays the total number of times this work’s files (e.g., a PDF) have been downloaded from QUT ePrints as well as the number of downloads in the previous 365 days. The count includes downloads for all files if a work has more than one.

ID Code: 82636
Item Type: Conference Paper
Refereed: Yes
Additional URLs:
Keywords: Facial expression, Heart rate, Video segmentation, Viewer interest, Sports video highlight
Divisions: Current > Schools > School of Electrical Engineering & Computer Science
Current > Institutes > Institute for Future Environments
Current > QUT Faculties and Divisions > Science & Engineering Faculty
Past > Schools > School of Information Systems
Copyright Owner: Copyright 2015 [please consult the authors]
Deposited On: 22 Mar 2015 22:39
Last Modified: 09 Jul 2015 18:20

Export: EndNote | Dublin Core | BibTeX

Repository Staff Only: item control page