Modeling Stress Using Thermal Facial Patterns: A Saptio-Temporal Approach

Nandita Sharma, Abhinav Dhall, Tom Gedeon, Roland GOECKE

Research output: A Conference proceeding or a Chapter in BookConference contribution

13 Citations (Scopus)

Abstract

Stress is a serious concern facing our world today, motivating the development of better objective understanding using non-intrusive means for stress recognition. The aim for the work was to use thermal imaging of facial regions to detect stress automatically. The work uses facial regions captured in videos in thermal (TS) and visible (VS) spectra and introduces ourdatabase ANU StressDB. It describes the experiment conducted for acquiring TS and VS videos of observers of stressed and not-stressed films for the ANU StressDB. Further, it presents an application of local binary patterns on three orthogonal planes (LBP-TOP) on VS and TS videos for stress recognition. It proposes a novel method to capture dynamic thermal patterns in histograms (HDTP) to utilise thermal and spatio-temporal characteristics associated in TS videos. Individual-independent support vector machine classifiers were developed for stress recognition. Results show that a fusion of facial patterns from VS and TS videos produced significantly better stress recognition rates than patterns from only VS or TS videos with p <0.01. The best stress recognition rate was 72% and it was obtained from HDTP features fused with LBP-TOP features for TS and VS videos, respectively
Original languageEnglish
Title of host publicationFifth Biannual Humaine Association Conference on Affective Computing and Intelligent Interaction
EditorsThierry Pun, Catherine Pelachaud, Nicu Sebe
Place of PublicationGeneva
PublisherIEEE, Institute of Electrical and Electronics Engineers
Pages387-392
Number of pages6
ISBN (Print)9780769550480
DOIs
Publication statusPublished - 2013
EventFifth Biannual Humaine Association Conference on Affective Computing and Intelligent Interaction: ACII 2013 - Emotion, Technology, Humanities - Geneva, Geneva, Switzerland
Duration: 2 Sep 20135 Sep 2013
http://Fifth Biannual Humaine Association Conference on Affective Computing and Intelligent Interaction (Conference Link)

Conference

ConferenceFifth Biannual Humaine Association Conference on Affective Computing and Intelligent Interaction
Abbreviated titleACII 2013
CountrySwitzerland
CityGeneva
Period2/09/135/09/13
Internet address

Fingerprint

Thermal stress
Infrared imaging
Support vector machines
Classifiers
Fusion reactions
Hot Temperature
Experiments

Cite this

Sharma, N., Dhall, A., Gedeon, T., & GOECKE, R. (2013). Modeling Stress Using Thermal Facial Patterns: A Saptio-Temporal Approach. In T. Pun, C. Pelachaud, & N. Sebe (Eds.), Fifth Biannual Humaine Association Conference on Affective Computing and Intelligent Interaction (pp. 387-392). Geneva: IEEE, Institute of Electrical and Electronics Engineers. https://doi.org/10.1109/ACII.2013.70
Sharma, Nandita ; Dhall, Abhinav ; Gedeon, Tom ; GOECKE, Roland. / Modeling Stress Using Thermal Facial Patterns: A Saptio-Temporal Approach. Fifth Biannual Humaine Association Conference on Affective Computing and Intelligent Interaction. editor / Thierry Pun ; Catherine Pelachaud ; Nicu Sebe. Geneva : IEEE, Institute of Electrical and Electronics Engineers, 2013. pp. 387-392
@inproceedings{454eeee0bfc5414e9417c6184da88ffa,
title = "Modeling Stress Using Thermal Facial Patterns: A Saptio-Temporal Approach",
abstract = "Stress is a serious concern facing our world today, motivating the development of better objective understanding using non-intrusive means for stress recognition. The aim for the work was to use thermal imaging of facial regions to detect stress automatically. The work uses facial regions captured in videos in thermal (TS) and visible (VS) spectra and introduces ourdatabase ANU StressDB. It describes the experiment conducted for acquiring TS and VS videos of observers of stressed and not-stressed films for the ANU StressDB. Further, it presents an application of local binary patterns on three orthogonal planes (LBP-TOP) on VS and TS videos for stress recognition. It proposes a novel method to capture dynamic thermal patterns in histograms (HDTP) to utilise thermal and spatio-temporal characteristics associated in TS videos. Individual-independent support vector machine classifiers were developed for stress recognition. Results show that a fusion of facial patterns from VS and TS videos produced significantly better stress recognition rates than patterns from only VS or TS videos with p <0.01. The best stress recognition rate was 72{\%} and it was obtained from HDTP features fused with LBP-TOP features for TS and VS videos, respectively",
keywords = "Emotion recognition, face analysis, Image classification, thermal imaging",
author = "Nandita Sharma and Abhinav Dhall and Tom Gedeon and Roland GOECKE",
year = "2013",
doi = "10.1109/ACII.2013.70",
language = "English",
isbn = "9780769550480",
pages = "387--392",
editor = "Thierry Pun and Catherine Pelachaud and Nicu Sebe",
booktitle = "Fifth Biannual Humaine Association Conference on Affective Computing and Intelligent Interaction",
publisher = "IEEE, Institute of Electrical and Electronics Engineers",
address = "United States",

}

Sharma, N, Dhall, A, Gedeon, T & GOECKE, R 2013, Modeling Stress Using Thermal Facial Patterns: A Saptio-Temporal Approach. in T Pun, C Pelachaud & N Sebe (eds), Fifth Biannual Humaine Association Conference on Affective Computing and Intelligent Interaction. IEEE, Institute of Electrical and Electronics Engineers, Geneva, pp. 387-392, Fifth Biannual Humaine Association Conference on Affective Computing and Intelligent Interaction, Geneva, Switzerland, 2/09/13. https://doi.org/10.1109/ACII.2013.70

Modeling Stress Using Thermal Facial Patterns: A Saptio-Temporal Approach. / Sharma, Nandita; Dhall, Abhinav; Gedeon, Tom; GOECKE, Roland.

Fifth Biannual Humaine Association Conference on Affective Computing and Intelligent Interaction. ed. / Thierry Pun; Catherine Pelachaud; Nicu Sebe. Geneva : IEEE, Institute of Electrical and Electronics Engineers, 2013. p. 387-392.

Research output: A Conference proceeding or a Chapter in BookConference contribution

TY - GEN

T1 - Modeling Stress Using Thermal Facial Patterns: A Saptio-Temporal Approach

AU - Sharma, Nandita

AU - Dhall, Abhinav

AU - Gedeon, Tom

AU - GOECKE, Roland

PY - 2013

Y1 - 2013

N2 - Stress is a serious concern facing our world today, motivating the development of better objective understanding using non-intrusive means for stress recognition. The aim for the work was to use thermal imaging of facial regions to detect stress automatically. The work uses facial regions captured in videos in thermal (TS) and visible (VS) spectra and introduces ourdatabase ANU StressDB. It describes the experiment conducted for acquiring TS and VS videos of observers of stressed and not-stressed films for the ANU StressDB. Further, it presents an application of local binary patterns on three orthogonal planes (LBP-TOP) on VS and TS videos for stress recognition. It proposes a novel method to capture dynamic thermal patterns in histograms (HDTP) to utilise thermal and spatio-temporal characteristics associated in TS videos. Individual-independent support vector machine classifiers were developed for stress recognition. Results show that a fusion of facial patterns from VS and TS videos produced significantly better stress recognition rates than patterns from only VS or TS videos with p <0.01. The best stress recognition rate was 72% and it was obtained from HDTP features fused with LBP-TOP features for TS and VS videos, respectively

AB - Stress is a serious concern facing our world today, motivating the development of better objective understanding using non-intrusive means for stress recognition. The aim for the work was to use thermal imaging of facial regions to detect stress automatically. The work uses facial regions captured in videos in thermal (TS) and visible (VS) spectra and introduces ourdatabase ANU StressDB. It describes the experiment conducted for acquiring TS and VS videos of observers of stressed and not-stressed films for the ANU StressDB. Further, it presents an application of local binary patterns on three orthogonal planes (LBP-TOP) on VS and TS videos for stress recognition. It proposes a novel method to capture dynamic thermal patterns in histograms (HDTP) to utilise thermal and spatio-temporal characteristics associated in TS videos. Individual-independent support vector machine classifiers were developed for stress recognition. Results show that a fusion of facial patterns from VS and TS videos produced significantly better stress recognition rates than patterns from only VS or TS videos with p <0.01. The best stress recognition rate was 72% and it was obtained from HDTP features fused with LBP-TOP features for TS and VS videos, respectively

KW - Emotion recognition

KW - face analysis

KW - Image classification

KW - thermal imaging

U2 - 10.1109/ACII.2013.70

DO - 10.1109/ACII.2013.70

M3 - Conference contribution

SN - 9780769550480

SP - 387

EP - 392

BT - Fifth Biannual Humaine Association Conference on Affective Computing and Intelligent Interaction

A2 - Pun, Thierry

A2 - Pelachaud, Catherine

A2 - Sebe, Nicu

PB - IEEE, Institute of Electrical and Electronics Engineers

CY - Geneva

ER -

Sharma N, Dhall A, Gedeon T, GOECKE R. Modeling Stress Using Thermal Facial Patterns: A Saptio-Temporal Approach. In Pun T, Pelachaud C, Sebe N, editors, Fifth Biannual Humaine Association Conference on Affective Computing and Intelligent Interaction. Geneva: IEEE, Institute of Electrical and Electronics Engineers. 2013. p. 387-392 https://doi.org/10.1109/ACII.2013.70