Heart Rate Estimation From Facial Videos for Depression Analysis

Aamir Mustafa; Shalini Bhatia; Munawar Hayat; Roland Goecke

doi:10.1109/ACII.2017.8273645

Heart Rate Estimation From Facial Videos for Depression Analysis

Aamir Mustafa, Shalini Bhatia, Munawar Hayat, Roland Goecke

Research output: A Conference proceeding or a Chapter in Book › Conference contribution

Abstract

Automated facial video analysis is useful in numerous health care applications. For example, spatio-temporal analysis of such videos has previously been done for assisting clinicians in the diagnosis of depression. Physiological measures, such as an individual’s heart rate, provide very important cues to understand a person’s mental health. Unobtrusively estimated heart rate has not been previously used to analyse an individuals’ mental health. In this paper, we automatically estimate heart rate activity from facial videos. We then study the association of the estimated heart rate activity with the person’s mental health, as diagnosed by clinicians. Specifically, from the heart rate activity in response
to watching different movies, we classify individuals as either depressed or healthy. The efficacy of the proposed scheme is demonstrated by experimental evaluations on a clinically validated dataset. Our results suggest unobtrusively estimated heart rate to be very effective for depression analysis.

Original language	English
Title of host publication	Proceedings of the 2017 Seventh International Conference on Affective Computing and Intelligent Interaction (ACII 2017)
Place of Publication	San Antonio, TX, USA
Publisher	IEEE, Institute of Electrical and Electronics Engineers
Pages	498-503
Number of pages	6
ISBN (Electronic)	9781538605639
ISBN (Print)	9781538605646
DOIs	https://doi.org/10.1109/ACII.2017.8273645
Publication status	Published - 23 Oct 2017

Publication series

Name	2017 SEVENTH INTERNATIONAL CONFERENCE ON AFFECTIVE COMPUTING AND INTELLIGENT INTERACTION (ACII)
ISSN (Print)	2156-8103

Access to Document

10.1109/ACII.2017.8273645

Link to IEEExplore page of publication

Cite this

Mustafa, A., Bhatia, S., Hayat, M., & Goecke, R. (2017). Heart Rate Estimation From Facial Videos for Depression Analysis. In Proceedings of the 2017 Seventh International Conference on Affective Computing and Intelligent Interaction (ACII 2017) (pp. 498-503). (2017 SEVENTH INTERNATIONAL CONFERENCE ON AFFECTIVE COMPUTING AND INTELLIGENT INTERACTION (ACII)). IEEE, Institute of Electrical and Electronics Engineers. https://doi.org/10.1109/ACII.2017.8273645

Mustafa, Aamir ; Bhatia, Shalini ; Hayat, Munawar ; Goecke, Roland. / Heart Rate Estimation From Facial Videos for Depression Analysis. Proceedings of the 2017 Seventh International Conference on Affective Computing and Intelligent Interaction (ACII 2017). San Antonio, TX, USA : IEEE, Institute of Electrical and Electronics Engineers, 2017. pp. 498-503 (2017 SEVENTH INTERNATIONAL CONFERENCE ON AFFECTIVE COMPUTING AND INTELLIGENT INTERACTION (ACII)).

@inproceedings{eb03c43366d64baab25ca9605f6c60a7,

title = "Heart Rate Estimation From Facial Videos for Depression Analysis",

abstract = "Automated facial video analysis is useful in numerous health care applications. For example, spatio-temporal analysis of such videos has previously been done for assisting clinicians in the diagnosis of depression. Physiological measures, such as an individual{\textquoteright}s heart rate, provide very important cues to understand a person{\textquoteright}s mental health. Unobtrusively estimated heart rate has not been previously used to analyse an individuals{\textquoteright} mental health. In this paper, we automatically estimate heart rate activity from facial videos. We then study the association of the estimated heart rate activity with the person{\textquoteright}s mental health, as diagnosed by clinicians. Specifically, from the heart rate activity in responseto watching different movies, we classify individuals as either depressed or healthy. The efficacy of the proposed scheme is demonstrated by experimental evaluations on a clinically validated dataset. Our results suggest unobtrusively estimated heart rate to be very effective for depression analysis.",

keywords = "Heart rate, Depression analysis, Video analysis, Physioloical measures",

author = "Aamir Mustafa and Shalini Bhatia and Munawar Hayat and Roland Goecke",

year = "2017",

month = oct

day = "23",

doi = "10.1109/ACII.2017.8273645",

language = "English",

isbn = "9781538605646",

series = "2017 SEVENTH INTERNATIONAL CONFERENCE ON AFFECTIVE COMPUTING AND INTELLIGENT INTERACTION (ACII)",

publisher = "IEEE, Institute of Electrical and Electronics Engineers",

pages = "498--503",

booktitle = "Proceedings of the 2017 Seventh International Conference on Affective Computing and Intelligent Interaction (ACII 2017)",

address = "United States",

}

Mustafa, A, Bhatia, S, Hayat, M & Goecke, R 2017, Heart Rate Estimation From Facial Videos for Depression Analysis. in Proceedings of the 2017 Seventh International Conference on Affective Computing and Intelligent Interaction (ACII 2017). 2017 SEVENTH INTERNATIONAL CONFERENCE ON AFFECTIVE COMPUTING AND INTELLIGENT INTERACTION (ACII), IEEE, Institute of Electrical and Electronics Engineers, San Antonio, TX, USA, pp. 498-503. https://doi.org/10.1109/ACII.2017.8273645

Heart Rate Estimation From Facial Videos for Depression Analysis. / Mustafa, Aamir; Bhatia, Shalini; Hayat, Munawar; Goecke, Roland.

Proceedings of the 2017 Seventh International Conference on Affective Computing and Intelligent Interaction (ACII 2017). San Antonio, TX, USA : IEEE, Institute of Electrical and Electronics Engineers, 2017. p. 498-503 (2017 SEVENTH INTERNATIONAL CONFERENCE ON AFFECTIVE COMPUTING AND INTELLIGENT INTERACTION (ACII)).

Research output: A Conference proceeding or a Chapter in Book › Conference contribution

TY - GEN

T1 - Heart Rate Estimation From Facial Videos for Depression Analysis

AU - Mustafa, Aamir

AU - Bhatia, Shalini

AU - Hayat, Munawar

AU - Goecke, Roland

PY - 2017/10/23

Y1 - 2017/10/23

N2 - Automated facial video analysis is useful in numerous health care applications. For example, spatio-temporal analysis of such videos has previously been done for assisting clinicians in the diagnosis of depression. Physiological measures, such as an individual’s heart rate, provide very important cues to understand a person’s mental health. Unobtrusively estimated heart rate has not been previously used to analyse an individuals’ mental health. In this paper, we automatically estimate heart rate activity from facial videos. We then study the association of the estimated heart rate activity with the person’s mental health, as diagnosed by clinicians. Specifically, from the heart rate activity in responseto watching different movies, we classify individuals as either depressed or healthy. The efficacy of the proposed scheme is demonstrated by experimental evaluations on a clinically validated dataset. Our results suggest unobtrusively estimated heart rate to be very effective for depression analysis.

AB - Automated facial video analysis is useful in numerous health care applications. For example, spatio-temporal analysis of such videos has previously been done for assisting clinicians in the diagnosis of depression. Physiological measures, such as an individual’s heart rate, provide very important cues to understand a person’s mental health. Unobtrusively estimated heart rate has not been previously used to analyse an individuals’ mental health. In this paper, we automatically estimate heart rate activity from facial videos. We then study the association of the estimated heart rate activity with the person’s mental health, as diagnosed by clinicians. Specifically, from the heart rate activity in responseto watching different movies, we classify individuals as either depressed or healthy. The efficacy of the proposed scheme is demonstrated by experimental evaluations on a clinically validated dataset. Our results suggest unobtrusively estimated heart rate to be very effective for depression analysis.

KW - Heart rate

KW - Depression analysis

KW - Video analysis

KW - Physioloical measures

UR - https://ieeexplore.ieee.org/document/8273645/

U2 - 10.1109/ACII.2017.8273645

DO - 10.1109/ACII.2017.8273645

M3 - Conference contribution

SN - 9781538605646

T3 - 2017 SEVENTH INTERNATIONAL CONFERENCE ON AFFECTIVE COMPUTING AND INTELLIGENT INTERACTION (ACII)

SP - 498

EP - 503

BT - Proceedings of the 2017 Seventh International Conference on Affective Computing and Intelligent Interaction (ACII 2017)

PB - IEEE, Institute of Electrical and Electronics Engineers

CY - San Antonio, TX, USA

ER -

Mustafa A, Bhatia S, Hayat M, Goecke R. Heart Rate Estimation From Facial Videos for Depression Analysis. In Proceedings of the 2017 Seventh International Conference on Affective Computing and Intelligent Interaction (ACII 2017). San Antonio, TX, USA: IEEE, Institute of Electrical and Electronics Engineers. 2017. p. 498-503. (2017 SEVENTH INTERNATIONAL CONFERENCE ON AFFECTIVE COMPUTING AND INTELLIGENT INTERACTION (ACII)). https://doi.org/10.1109/ACII.2017.8273645