Diagnosis of depression by behavioural signals: A multimodal approach

Nicholas Cummins, Jyoti Joshi, Abhinav Dhall, Vidhyasaharan Sethu, Roland Goecke, Julien Epps

Research output: A Conference proceeding or a Chapter in BookConference contributionpeer-review

105 Citations (Scopus)


Quantifying behavioural changes in depression using affective computing techniques is the first step in developing an objective diagnostic aid, with clinical utility, for clinical depression. As part of the AVEC 2013 Challenge, we present a multimodal approach for the Depression Sub-Challenge using a GMM-UBM system with three different kernels for the audio subsystem and Space Time Interest Points in a Bag-of-Words approach for the vision subsystem. These are then fused at the feature level to form the combined AV system. Key results include the strong performance of acoustic audio features and the bag-of-words visual features in predicting an individual’s level of depression using regression. Interestingly, in the context of the small amount of literature on the subject, is that our feature level multimodal fusion technique is able to outperform both the audio and visual challenge baselines.
Original languageEnglish
Title of host publicationAVEC 2013 - Proceedings of the 3rd ACM International Workshop on Audio/Visual Emotion Challenge
EditorsBjorn Schuller, Michel Valstar, Roddy Cowie, Maja Pantic, Jarek Krajewski
Place of PublicationBarcelona, Spain
PublisherAssociation for Computing Machinery (ACM)
Number of pages10
ISBN (Print)9781450323956
Publication statusPublished - 2013
EventACM International Workshop on Audio/Visual Emotion Challenge - Barcelona, Barcelona, Spain
Duration: 21 Oct 201325 Oct 2013


ConferenceACM International Workshop on Audio/Visual Emotion Challenge


Dive into the research topics of 'Diagnosis of depression by behavioural signals: A multimodal approach'. Together they form a unique fingerprint.

Cite this