The modeling of human facial pain intensity based on Temporal Convolutional Networks trained with video frames in HSV color space

Ghazal Bargshady, Xujuan Zhou, Ravinesh C. Deo, Jeffrey Soar, Frank Whittaker, Hua Wang

Research output: Contribution to journalArticlepeer-review

27 Citations (Scopus)

Abstract

An accurate detection and management of pain, measured through its relative intensity, plays an important role in the treatment of disease and reducing a patient's discomfort. As it is relatively difficult to assess, describe, evaluate and manage the pain level using a patient's self-report, automated pain-detecting tools can provide useful information to assist in the management of pain intensity. This study proposes a new predictive modeling framework that employs a modified Temporal Convolutional Network (TCN) algorithm to recognize the pain intensity prevalent in patients’ video frames collected as part of UNBC-McMaster Shoulder Pain Archive and MIntPAIN databases. The inputs of the proposed TCN network is composed of the extracted and reduced face image features from a fine-tuned VGG-Face and principal component analysis (PCA) with Hue, Saturation, Value (HSV) color spaces video images. The results of TCN based predictive model, employing a long short-term memory (LSTM) model as well as other state-of-the art models, show that the proposed approach performs faster with a high level of efficiency. This is demonstrated by the low magnitude of error metrics (i.e., Mean Squared Error = 0.0629, Mean Absolute Error = 0.1021, correctness validation results represented by Area under Curve = 85% and accuracy metric = 92.44%). Considering the efficiency of the proposed TCN framework, integrating fine-tuned VGG-Face and PCA with Hue, Saturation, Value (HSV) color spaces video images for pain intensity estimation, the present study affirms that the new method can be adopted as an automatic health informatics tool, mainly for pain detection, and subsequently, implemented in the pain management area.

Original languageEnglish
Article number106805
Pages (from-to)1-14
Number of pages14
JournalApplied Soft Computing Journal
Volume97
DOIs
Publication statusPublished - Dec 2020
Externally publishedYes

Fingerprint

Dive into the research topics of 'The modeling of human facial pain intensity based on Temporal Convolutional Networks trained with video frames in HSV color space'. Together they form a unique fingerprint.

Cite this