Investigating word affect features and fusion of probabilistic predictions incorporating uncertainty in AVEC 2017

Ting Dang, Brian Stasak, Zhaocheng Huang, Sadari Jayawardena, Mia Atcheson, Munawar Hayat, Phu Le, Vidhyasaharan Sethu, Roland Goecke, Julien Epps

Research output: A Conference proceeding or a Chapter in BookConference contribution

10 Citations (Scopus)

Abstract

Predicting emotion intensity and severity of depression are both challenging and important problems within the broader field of affective computing. As part of the AVEC 2017, we developed a number of systems to accomplish these tasks. In particular, word affect features, which derive human affect ratings (e.g. arousal and valence) from transcripts, were investigated for predicting depression severity and liking, showing great promise. A simple system based on the word affect features achieved an RMSE of 6.02 on the test set, yielding a relative improvement of 13.6% over the baseline. For the emotion prediction sub-challenge, we investigated multimodal fusion, which incorporated a measure of uncertainty associated with each prediction within an Output-Associative fusion framework for arousal and valence prediction, whilst liking prediction systems mainly focused on text-based features. Our best emotion prediction systems provided significant relative improvements over the baseline on the test set of 39.5%, 17.6%, and 29.3% for arousal, valence, and liking. Of particular note is that consistent improvements were observed when incorporating prediction uncertainty across various system configurations for predicting arousal and valence, suggesting the importance of taking into consideration prediction uncertainty for fusion and more broadly the advantages of probabilistic predictions.

Original languageEnglish
Title of host publicationAVEC 2017 - Proceedings of the 7th Annual Workshop on Audio/Visual Emotion Challenge, co-located with MM 2017
Place of PublicationMountain View, USA
PublisherACM Association for Computing Machinery
Pages27-35
Number of pages9
ISBN (Electronic)9781450355025
DOIs
Publication statusPublished - 23 Oct 2017
Event7th Annual Workshop on Audio/Visual Emotion Challenge, AVEC 2017 - Mountain View, United States
Duration: 23 Oct 2017 → …

Publication series

NameAVEC 2017 - Proceedings of the 7th Annual Workshop on Audio/Visual Emotion Challenge, co-located with MM 2017

Conference

Conference7th Annual Workshop on Audio/Visual Emotion Challenge, AVEC 2017
CountryUnited States
CityMountain View
Period23/10/17 → …

Fingerprint Dive into the research topics of 'Investigating word affect features and fusion of probabilistic predictions incorporating uncertainty in AVEC 2017'. Together they form a unique fingerprint.

  • Cite this

    Dang, T., Stasak, B., Huang, Z., Jayawardena, S., Atcheson, M., Hayat, M., Le, P., Sethu, V., Goecke, R., & Epps, J. (2017). Investigating word affect features and fusion of probabilistic predictions incorporating uncertainty in AVEC 2017. In AVEC 2017 - Proceedings of the 7th Annual Workshop on Audio/Visual Emotion Challenge, co-located with MM 2017 (pp. 27-35). (AVEC 2017 - Proceedings of the 7th Annual Workshop on Audio/Visual Emotion Challenge, co-located with MM 2017). ACM Association for Computing Machinery. https://doi.org/10.1145/3133944.3133952