Can Body Expressions Contribute to Automatic Depression Analysis?

Jyoti Joshi, Roland GOECKE, Gordon Parker, Michael Breakspear

    Research output: A Conference proceeding or a Chapter in BookConference contribution

    33 Citations (Scopus)
    5 Downloads (Pure)

    Abstract

    Depression is one of the most common mental health disorders with strong adverse effects on personal and social functioning. The absence of any objective diagnostic aid for depression leads to a range of subjective biases in initial diagnosis and ongoing monitoring. Psychologists use various visual cues in their assessment to quantify depression such as facial expressions, eye contact and head movements. This paper studies the contribution of (upper) body expressions and gestures for automatic depression analysis. A framework based on space-time interest points and bag of words is proposed for the analysis of upper body and facial movements. Salient interest points are selected using clustering. The major contribution of this paper lies in the creation of a bag of body expressions and a bag of facial dynamics for assessing the contribution of different body parts for depression analysis. Head movement analysis is performed by selecting rigid facial fiducial points and a new histogram of head movements is proposed. The experiments are performed on real-world clinical data where video clips of patients and healthy controls are recorded during interactive interview sessions. The results show the effectiveness of the proposed system to evaluate the contribution of various body parts in depression analysis.
    Original languageEnglish
    Title of host publicationIEEE International Conference on Automated Face and Gesture Recognition FG2013
    EditorsRama Chellappa, Xilin Chen, Qiang Ji
    Place of PublicationShanghai, China
    PublisherIEEE, Institute of Electrical and Electronics Engineers
    Pages1-7
    Number of pages7
    Volume1
    ISBN (Print)9781467355452
    DOIs
    Publication statusPublished - 2013
    EventIEEE International Conference on Automated Face and Gesture Recognition FG2013 - Shanghai, Shanghai, China
    Duration: 22 Apr 201326 Apr 2013

    Conference

    ConferenceIEEE International Conference on Automated Face and Gesture Recognition FG2013
    CountryChina
    CityShanghai
    Period22/04/1326/04/13

    Fingerprint

    Health
    Monitoring
    Experiments

    Cite this

    Joshi, J., GOECKE, R., Parker, G., & Breakspear, M. (2013). Can Body Expressions Contribute to Automatic Depression Analysis? In R. Chellappa, X. Chen, & Q. Ji (Eds.), IEEE International Conference on Automated Face and Gesture Recognition FG2013 (Vol. 1, pp. 1-7). Shanghai, China: IEEE, Institute of Electrical and Electronics Engineers. https://doi.org/10.1109/FG.2013.6553796
    Joshi, Jyoti ; GOECKE, Roland ; Parker, Gordon ; Breakspear, Michael. / Can Body Expressions Contribute to Automatic Depression Analysis?. IEEE International Conference on Automated Face and Gesture Recognition FG2013. editor / Rama Chellappa ; Xilin Chen ; Qiang Ji. Vol. 1 Shanghai, China : IEEE, Institute of Electrical and Electronics Engineers, 2013. pp. 1-7
    @inproceedings{d8d4cbb2427f4d25b8f74c2e677e685e,
    title = "Can Body Expressions Contribute to Automatic Depression Analysis?",
    abstract = "Depression is one of the most common mental health disorders with strong adverse effects on personal and social functioning. The absence of any objective diagnostic aid for depression leads to a range of subjective biases in initial diagnosis and ongoing monitoring. Psychologists use various visual cues in their assessment to quantify depression such as facial expressions, eye contact and head movements. This paper studies the contribution of (upper) body expressions and gestures for automatic depression analysis. A framework based on space-time interest points and bag of words is proposed for the analysis of upper body and facial movements. Salient interest points are selected using clustering. The major contribution of this paper lies in the creation of a bag of body expressions and a bag of facial dynamics for assessing the contribution of different body parts for depression analysis. Head movement analysis is performed by selecting rigid facial fiducial points and a new histogram of head movements is proposed. The experiments are performed on real-world clinical data where video clips of patients and healthy controls are recorded during interactive interview sessions. The results show the effectiveness of the proposed system to evaluate the contribution of various body parts in depression analysis.",
    keywords = "face analysis, gesture recognition, Depression analysis, body expressions",
    author = "Jyoti Joshi and Roland GOECKE and Gordon Parker and Michael Breakspear",
    year = "2013",
    doi = "10.1109/FG.2013.6553796",
    language = "English",
    isbn = "9781467355452",
    volume = "1",
    pages = "1--7",
    editor = "Rama Chellappa and Xilin Chen and Qiang Ji",
    booktitle = "IEEE International Conference on Automated Face and Gesture Recognition FG2013",
    publisher = "IEEE, Institute of Electrical and Electronics Engineers",
    address = "United States",

    }

    Joshi, J, GOECKE, R, Parker, G & Breakspear, M 2013, Can Body Expressions Contribute to Automatic Depression Analysis? in R Chellappa, X Chen & Q Ji (eds), IEEE International Conference on Automated Face and Gesture Recognition FG2013. vol. 1, IEEE, Institute of Electrical and Electronics Engineers, Shanghai, China, pp. 1-7, IEEE International Conference on Automated Face and Gesture Recognition FG2013, Shanghai, China, 22/04/13. https://doi.org/10.1109/FG.2013.6553796

    Can Body Expressions Contribute to Automatic Depression Analysis? / Joshi, Jyoti; GOECKE, Roland; Parker, Gordon; Breakspear, Michael.

    IEEE International Conference on Automated Face and Gesture Recognition FG2013. ed. / Rama Chellappa; Xilin Chen; Qiang Ji. Vol. 1 Shanghai, China : IEEE, Institute of Electrical and Electronics Engineers, 2013. p. 1-7.

    Research output: A Conference proceeding or a Chapter in BookConference contribution

    TY - GEN

    T1 - Can Body Expressions Contribute to Automatic Depression Analysis?

    AU - Joshi, Jyoti

    AU - GOECKE, Roland

    AU - Parker, Gordon

    AU - Breakspear, Michael

    PY - 2013

    Y1 - 2013

    N2 - Depression is one of the most common mental health disorders with strong adverse effects on personal and social functioning. The absence of any objective diagnostic aid for depression leads to a range of subjective biases in initial diagnosis and ongoing monitoring. Psychologists use various visual cues in their assessment to quantify depression such as facial expressions, eye contact and head movements. This paper studies the contribution of (upper) body expressions and gestures for automatic depression analysis. A framework based on space-time interest points and bag of words is proposed for the analysis of upper body and facial movements. Salient interest points are selected using clustering. The major contribution of this paper lies in the creation of a bag of body expressions and a bag of facial dynamics for assessing the contribution of different body parts for depression analysis. Head movement analysis is performed by selecting rigid facial fiducial points and a new histogram of head movements is proposed. The experiments are performed on real-world clinical data where video clips of patients and healthy controls are recorded during interactive interview sessions. The results show the effectiveness of the proposed system to evaluate the contribution of various body parts in depression analysis.

    AB - Depression is one of the most common mental health disorders with strong adverse effects on personal and social functioning. The absence of any objective diagnostic aid for depression leads to a range of subjective biases in initial diagnosis and ongoing monitoring. Psychologists use various visual cues in their assessment to quantify depression such as facial expressions, eye contact and head movements. This paper studies the contribution of (upper) body expressions and gestures for automatic depression analysis. A framework based on space-time interest points and bag of words is proposed for the analysis of upper body and facial movements. Salient interest points are selected using clustering. The major contribution of this paper lies in the creation of a bag of body expressions and a bag of facial dynamics for assessing the contribution of different body parts for depression analysis. Head movement analysis is performed by selecting rigid facial fiducial points and a new histogram of head movements is proposed. The experiments are performed on real-world clinical data where video clips of patients and healthy controls are recorded during interactive interview sessions. The results show the effectiveness of the proposed system to evaluate the contribution of various body parts in depression analysis.

    KW - face analysis

    KW - gesture recognition

    KW - Depression analysis

    KW - body expressions

    U2 - 10.1109/FG.2013.6553796

    DO - 10.1109/FG.2013.6553796

    M3 - Conference contribution

    SN - 9781467355452

    VL - 1

    SP - 1

    EP - 7

    BT - IEEE International Conference on Automated Face and Gesture Recognition FG2013

    A2 - Chellappa, Rama

    A2 - Chen, Xilin

    A2 - Ji, Qiang

    PB - IEEE, Institute of Electrical and Electronics Engineers

    CY - Shanghai, China

    ER -

    Joshi J, GOECKE R, Parker G, Breakspear M. Can Body Expressions Contribute to Automatic Depression Analysis? In Chellappa R, Chen X, Ji Q, editors, IEEE International Conference on Automated Face and Gesture Recognition FG2013. Vol. 1. Shanghai, China: IEEE, Institute of Electrical and Electronics Engineers. 2013. p. 1-7 https://doi.org/10.1109/FG.2013.6553796