Investigating word affect features and fusion of probabilistic predictions incorporating uncertainty in AVEC 2017

Ting Dang, Brian Stasak, Zhaocheng Huang, Sadari Jayawardena, Mia Atcheson, Munawar Hayat, Phu Le, Vidhyasaharan Sethu, Roland Goecke, Julien Epps

Research output: A Conference proceeding or a Chapter in BookConference contribution

6 Citations (Scopus)

Abstract

Predicting emotion intensity and severity of depression are both challenging and important problems within the broader field of affective computing. As part of the AVEC 2017, we developed a number of systems to accomplish these tasks. In particular, word affect features, which derive human affect ratings (e.g. arousal and valence) from transcripts, were investigated for predicting depression severity and liking, showing great promise. A simple system based on the word affect features achieved an RMSE of 6.02 on the test set, yielding a relative improvement of 13.6% over the baseline. For the emotion prediction sub-challenge, we investigated multimodal fusion, which incorporated a measure of uncertainty associated with each prediction within an Output-Associative fusion framework for arousal and valence prediction, whilst liking prediction systems mainly focused on text-based features. Our best emotion prediction systems provided significant relative improvements over the baseline on the test set of 39.5%, 17.6%, and 29.3% for arousal, valence, and liking. Of particular note is that consistent improvements were observed when incorporating prediction uncertainty across various system configurations for predicting arousal and valence, suggesting the importance of taking into consideration prediction uncertainty for fusion and more broadly the advantages of probabilistic predictions.

Original languageEnglish
Title of host publicationAVEC 2017 - Proceedings of the 7th Annual Workshop on Audio/Visual Emotion Challenge, co-located with MM 2017
Place of PublicationMountain View, USA
PublisherACM Association for Computing Machinery
Pages27-35
Number of pages9
ISBN (Electronic)9781450355025
DOIs
Publication statusPublished - 23 Oct 2017
Event7th Annual Workshop on Audio/Visual Emotion Challenge, AVEC 2017 - Mountain View, United States
Duration: 23 Oct 2017 → …

Conference

Conference7th Annual Workshop on Audio/Visual Emotion Challenge, AVEC 2017
CountryUnited States
CityMountain View
Period23/10/17 → …

Fingerprint

Fusion reactions
Uncertainty

Cite this

Dang, T., Stasak, B., Huang, Z., Jayawardena, S., Atcheson, M., Hayat, M., ... Epps, J. (2017). Investigating word affect features and fusion of probabilistic predictions incorporating uncertainty in AVEC 2017. In AVEC 2017 - Proceedings of the 7th Annual Workshop on Audio/Visual Emotion Challenge, co-located with MM 2017 (pp. 27-35). Mountain View, USA: ACM Association for Computing Machinery. https://doi.org/10.1145/3133944.3133952
Dang, Ting ; Stasak, Brian ; Huang, Zhaocheng ; Jayawardena, Sadari ; Atcheson, Mia ; Hayat, Munawar ; Le, Phu ; Sethu, Vidhyasaharan ; Goecke, Roland ; Epps, Julien. / Investigating word affect features and fusion of probabilistic predictions incorporating uncertainty in AVEC 2017. AVEC 2017 - Proceedings of the 7th Annual Workshop on Audio/Visual Emotion Challenge, co-located with MM 2017. Mountain View, USA : ACM Association for Computing Machinery, 2017. pp. 27-35
@inproceedings{970189b6b730434f81f3c327c31b2114,
title = "Investigating word affect features and fusion of probabilistic predictions incorporating uncertainty in AVEC 2017",
abstract = "Predicting emotion intensity and severity of depression are both challenging and important problems within the broader field of affective computing. As part of the AVEC 2017, we developed a number of systems to accomplish these tasks. In particular, word affect features, which derive human affect ratings (e.g. arousal and valence) from transcripts, were investigated for predicting depression severity and liking, showing great promise. A simple system based on the word affect features achieved an RMSE of 6.02 on the test set, yielding a relative improvement of 13.6{\%} over the baseline. For the emotion prediction sub-challenge, we investigated multimodal fusion, which incorporated a measure of uncertainty associated with each prediction within an Output-Associative fusion framework for arousal and valence prediction, whilst liking prediction systems mainly focused on text-based features. Our best emotion prediction systems provided significant relative improvements over the baseline on the test set of 39.5{\%}, 17.6{\%}, and 29.3{\%} for arousal, valence, and liking. Of particular note is that consistent improvements were observed when incorporating prediction uncertainty across various system configurations for predicting arousal and valence, suggesting the importance of taking into consideration prediction uncertainty for fusion and more broadly the advantages of probabilistic predictions.",
keywords = "Depression prediction, Dimensional emotion prediction, Output-associative fusion, Sentiment analysis, Uncertainty prediction",
author = "Ting Dang and Brian Stasak and Zhaocheng Huang and Sadari Jayawardena and Mia Atcheson and Munawar Hayat and Phu Le and Vidhyasaharan Sethu and Roland Goecke and Julien Epps",
year = "2017",
month = "10",
day = "23",
doi = "10.1145/3133944.3133952",
language = "English",
pages = "27--35",
booktitle = "AVEC 2017 - Proceedings of the 7th Annual Workshop on Audio/Visual Emotion Challenge, co-located with MM 2017",
publisher = "ACM Association for Computing Machinery",

}

Dang, T, Stasak, B, Huang, Z, Jayawardena, S, Atcheson, M, Hayat, M, Le, P, Sethu, V, Goecke, R & Epps, J 2017, Investigating word affect features and fusion of probabilistic predictions incorporating uncertainty in AVEC 2017. in AVEC 2017 - Proceedings of the 7th Annual Workshop on Audio/Visual Emotion Challenge, co-located with MM 2017. ACM Association for Computing Machinery, Mountain View, USA, pp. 27-35, 7th Annual Workshop on Audio/Visual Emotion Challenge, AVEC 2017, Mountain View, United States, 23/10/17. https://doi.org/10.1145/3133944.3133952

Investigating word affect features and fusion of probabilistic predictions incorporating uncertainty in AVEC 2017. / Dang, Ting; Stasak, Brian; Huang, Zhaocheng; Jayawardena, Sadari; Atcheson, Mia; Hayat, Munawar; Le, Phu; Sethu, Vidhyasaharan; Goecke, Roland; Epps, Julien.

AVEC 2017 - Proceedings of the 7th Annual Workshop on Audio/Visual Emotion Challenge, co-located with MM 2017. Mountain View, USA : ACM Association for Computing Machinery, 2017. p. 27-35.

Research output: A Conference proceeding or a Chapter in BookConference contribution

TY - GEN

T1 - Investigating word affect features and fusion of probabilistic predictions incorporating uncertainty in AVEC 2017

AU - Dang, Ting

AU - Stasak, Brian

AU - Huang, Zhaocheng

AU - Jayawardena, Sadari

AU - Atcheson, Mia

AU - Hayat, Munawar

AU - Le, Phu

AU - Sethu, Vidhyasaharan

AU - Goecke, Roland

AU - Epps, Julien

PY - 2017/10/23

Y1 - 2017/10/23

N2 - Predicting emotion intensity and severity of depression are both challenging and important problems within the broader field of affective computing. As part of the AVEC 2017, we developed a number of systems to accomplish these tasks. In particular, word affect features, which derive human affect ratings (e.g. arousal and valence) from transcripts, were investigated for predicting depression severity and liking, showing great promise. A simple system based on the word affect features achieved an RMSE of 6.02 on the test set, yielding a relative improvement of 13.6% over the baseline. For the emotion prediction sub-challenge, we investigated multimodal fusion, which incorporated a measure of uncertainty associated with each prediction within an Output-Associative fusion framework for arousal and valence prediction, whilst liking prediction systems mainly focused on text-based features. Our best emotion prediction systems provided significant relative improvements over the baseline on the test set of 39.5%, 17.6%, and 29.3% for arousal, valence, and liking. Of particular note is that consistent improvements were observed when incorporating prediction uncertainty across various system configurations for predicting arousal and valence, suggesting the importance of taking into consideration prediction uncertainty for fusion and more broadly the advantages of probabilistic predictions.

AB - Predicting emotion intensity and severity of depression are both challenging and important problems within the broader field of affective computing. As part of the AVEC 2017, we developed a number of systems to accomplish these tasks. In particular, word affect features, which derive human affect ratings (e.g. arousal and valence) from transcripts, were investigated for predicting depression severity and liking, showing great promise. A simple system based on the word affect features achieved an RMSE of 6.02 on the test set, yielding a relative improvement of 13.6% over the baseline. For the emotion prediction sub-challenge, we investigated multimodal fusion, which incorporated a measure of uncertainty associated with each prediction within an Output-Associative fusion framework for arousal and valence prediction, whilst liking prediction systems mainly focused on text-based features. Our best emotion prediction systems provided significant relative improvements over the baseline on the test set of 39.5%, 17.6%, and 29.3% for arousal, valence, and liking. Of particular note is that consistent improvements were observed when incorporating prediction uncertainty across various system configurations for predicting arousal and valence, suggesting the importance of taking into consideration prediction uncertainty for fusion and more broadly the advantages of probabilistic predictions.

KW - Depression prediction

KW - Dimensional emotion prediction

KW - Output-associative fusion

KW - Sentiment analysis

KW - Uncertainty prediction

UR - http://www.scopus.com/inward/record.url?scp=85035314372&partnerID=8YFLogxK

U2 - 10.1145/3133944.3133952

DO - 10.1145/3133944.3133952

M3 - Conference contribution

SP - 27

EP - 35

BT - AVEC 2017 - Proceedings of the 7th Annual Workshop on Audio/Visual Emotion Challenge, co-located with MM 2017

PB - ACM Association for Computing Machinery

CY - Mountain View, USA

ER -

Dang T, Stasak B, Huang Z, Jayawardena S, Atcheson M, Hayat M et al. Investigating word affect features and fusion of probabilistic predictions incorporating uncertainty in AVEC 2017. In AVEC 2017 - Proceedings of the 7th Annual Workshop on Audio/Visual Emotion Challenge, co-located with MM 2017. Mountain View, USA: ACM Association for Computing Machinery. 2017. p. 27-35 https://doi.org/10.1145/3133944.3133952