TY - JOUR
T1 - The modeling of human facial pain intensity based on Temporal Convolutional Networks trained with video frames in HSV color space
AU - Bargshady, Ghazal
AU - Zhou, Xujuan
AU - Deo, Ravinesh C.
AU - Soar, Jeffrey
AU - Whittaker, Frank
AU - Wang, Hua
N1 - Funding Information:
This research has been supported by an Australian Government Research Training Program Scholarship and was funded by the Australian Research Council (ARC) (grant number LP150100673) supported by Nexus eCare Pty Ltd .
Publisher Copyright:
© 2020 Elsevier B.V.
PY - 2020/12
Y1 - 2020/12
N2 - An accurate detection and management of pain, measured through its relative intensity, plays an important role in the treatment of disease and reducing a patient's discomfort. As it is relatively difficult to assess, describe, evaluate and manage the pain level using a patient's self-report, automated pain-detecting tools can provide useful information to assist in the management of pain intensity. This study proposes a new predictive modeling framework that employs a modified Temporal Convolutional Network (TCN) algorithm to recognize the pain intensity prevalent in patients’ video frames collected as part of UNBC-McMaster Shoulder Pain Archive and MIntPAIN databases. The inputs of the proposed TCN network is composed of the extracted and reduced face image features from a fine-tuned VGG-Face and principal component analysis (PCA) with Hue, Saturation, Value (HSV) color spaces video images. The results of TCN based predictive model, employing a long short-term memory (LSTM) model as well as other state-of-the art models, show that the proposed approach performs faster with a high level of efficiency. This is demonstrated by the low magnitude of error metrics (i.e., Mean Squared Error = 0.0629, Mean Absolute Error = 0.1021, correctness validation results represented by Area under Curve = 85% and accuracy metric = 92.44%). Considering the efficiency of the proposed TCN framework, integrating fine-tuned VGG-Face and PCA with Hue, Saturation, Value (HSV) color spaces video images for pain intensity estimation, the present study affirms that the new method can be adopted as an automatic health informatics tool, mainly for pain detection, and subsequently, implemented in the pain management area.
AB - An accurate detection and management of pain, measured through its relative intensity, plays an important role in the treatment of disease and reducing a patient's discomfort. As it is relatively difficult to assess, describe, evaluate and manage the pain level using a patient's self-report, automated pain-detecting tools can provide useful information to assist in the management of pain intensity. This study proposes a new predictive modeling framework that employs a modified Temporal Convolutional Network (TCN) algorithm to recognize the pain intensity prevalent in patients’ video frames collected as part of UNBC-McMaster Shoulder Pain Archive and MIntPAIN databases. The inputs of the proposed TCN network is composed of the extracted and reduced face image features from a fine-tuned VGG-Face and principal component analysis (PCA) with Hue, Saturation, Value (HSV) color spaces video images. The results of TCN based predictive model, employing a long short-term memory (LSTM) model as well as other state-of-the art models, show that the proposed approach performs faster with a high level of efficiency. This is demonstrated by the low magnitude of error metrics (i.e., Mean Squared Error = 0.0629, Mean Absolute Error = 0.1021, correctness validation results represented by Area under Curve = 85% and accuracy metric = 92.44%). Considering the efficiency of the proposed TCN framework, integrating fine-tuned VGG-Face and PCA with Hue, Saturation, Value (HSV) color spaces video images for pain intensity estimation, the present study affirms that the new method can be adopted as an automatic health informatics tool, mainly for pain detection, and subsequently, implemented in the pain management area.
KW - Facial expression
KW - HSV color space
KW - Pain detection
KW - Temporal convolutional network
KW - Video analysis
UR - http://www.scopus.com/inward/record.url?scp=85092935614&partnerID=8YFLogxK
U2 - 10.1016/j.asoc.2020.106805
DO - 10.1016/j.asoc.2020.106805
M3 - Article
AN - SCOPUS:85092935614
SN - 1568-4946
VL - 97
SP - 1
EP - 14
JO - Applied Soft Computing Journal
JF - Applied Soft Computing Journal
M1 - 106805
ER -