TY - GEN
T1 - A multimodal system to characterise melancholia: Cascaded bag of words approach
AU - Bhatia, Shalini
AU - Hayat, Munawar
AU - Goecke, Roland
PY - 2017/11/3
Y1 - 2017/11/3
N2 - Recent years have seen a lot of activity in affective computing for automated analysis of depression. However, no research has so far proposed a multimodal system for classifying different subtypes of depression such as melancholia. The mental state assessment of a mood disorder depends primarily on appearance, behaviour, speech, thought, perception, mood and facial affect. Mood and facial affect mainly contribute to distinguishing melancholia from nonmelancholia. These are assessed by clinicians, and hence vulnerable to subjective judgement. As a result, clinical assessment alone may not accurately capture the presence or absence of specific disorders such as melancholia, a distressing condition whose presence has important treatment implications. Melancholia is characterised by severe anhedonia and psychomotor disturbance, which can be a combination of motor retardation with periods of superimposed agitation. Psychomotor disturbance can be sensed in both face and voice. To the best of our knowledge, this study is the first attempt to propose a multimodal system to differentiate melancholia from non-melancholia and healthy controls. We report the sensitivity and specificity of classification in depressive subtypes.
AB - Recent years have seen a lot of activity in affective computing for automated analysis of depression. However, no research has so far proposed a multimodal system for classifying different subtypes of depression such as melancholia. The mental state assessment of a mood disorder depends primarily on appearance, behaviour, speech, thought, perception, mood and facial affect. Mood and facial affect mainly contribute to distinguishing melancholia from nonmelancholia. These are assessed by clinicians, and hence vulnerable to subjective judgement. As a result, clinical assessment alone may not accurately capture the presence or absence of specific disorders such as melancholia, a distressing condition whose presence has important treatment implications. Melancholia is characterised by severe anhedonia and psychomotor disturbance, which can be a combination of motor retardation with periods of superimposed agitation. Psychomotor disturbance can be sensed in both face and voice. To the best of our knowledge, this study is the first attempt to propose a multimodal system to differentiate melancholia from non-melancholia and healthy controls. We report the sensitivity and specificity of classification in depressive subtypes.
KW - Audio-Video
KW - Bag of words
KW - Multimodal fusion
KW - Melancholia
KW - Depression analysis
KW - Multimodal
KW - Video
KW - Audio
KW - Fusion
UR - http://www.scopus.com/inward/record.url?scp=85046636068&partnerID=8YFLogxK
UR - https://dl.acm.org/citation.cfm?doid=3136755.3136766
UR - http://www.mendeley.com/research/multimodal-system-characterise-melancholia-cascaded-bag-words-approach
U2 - 10.1145/3136755.3136766
DO - 10.1145/3136755.3136766
M3 - Conference contribution
AN - SCOPUS:85046636068
SN - 9781450355438
T3 - ICMI 2017 - Proceedings of the 19th ACM International Conference on Multimodal Interaction
SP - 274
EP - 280
BT - ICMI 2017 - Proceedings of the 19th ACM International Conference on Multimodal Interaction
PB - Association for Computing Machinery, Inc
CY - Glasgow, United Kingdom
T2 - 19th ACM International Conference on Multimodal Interaction, ICMI 2017
Y2 - 13 November 2017 through 17 November 2017
ER -