To Improve Is to Change: Towards Improving Mood Prediction by Learning Changes in Emotion

Soujanya Narayana; Ramanathan Subramanian; Ibrahim Radwan; Roland Goecke

doi:10.1145/3536220.3563685

To Improve Is to Change: Towards Improving Mood Prediction by Learning Changes in Emotion

Soujanya Narayana, Ramanathan Subramanian, Ibrahim Radwan, Roland Goecke

Information Systems

Research output: A Conference proceeding or a Chapter in Book › Conference contribution › peer-review

3 Citations (Scopus)

Abstract

Although the terms mood and emotion are closely related and often used interchangeably, they are distinguished based on their duration, intensity and attribution. To date, hardly any computational models have (a) examined mood recognition, and (b) modelled the interplay between mood and emotional state in their analysis. In this paper, as a first step towards mood prediction, we propose a framework that utilises both dominant emotion (or mood) labels, and emotional change labels on the AFEW-VA database. Experiments evaluating unimodal (trained only using mood labels) and multimodal (trained with both mood and emotion change labels) convolutional neural networks confirm that incorporating emotional change information in the network training process can significantly improve the mood prediction performance, thus highlighting the importance of modelling emotion and mood simultaneously for improved performance in affective state recognition.

Original language	English
Title of host publication	ICMI 2022 - Companion Publication of the 2022 International Conference on Multimodal Interaction
Editors	Raj Tumuluri, Nicu Sebe, Gopal Pingali, Dinesh Babu Jayagopi
Place of Publication	New York
Publisher	Association for Computing Machinery (ACM)
Pages	36-41
Number of pages	6
ISBN (Electronic)	9781450393898
ISBN (Print)	9781450393898
DOIs	https://doi.org/10.1145/3536220.3563685
Publication status	Published - 7 Nov 2022
Event	24th ACM International Conference on Multimodal Interaction, ICMI 2022 - Bangalore, India Duration: 7 Nov 2022 → 11 Nov 2022 https://icmi.acm.org/2022/

Publication series

Name	ACM International Conference Proceeding Series

Conference

Conference	24th ACM International Conference on Multimodal Interaction, ICMI 2022
Abbreviated title	ICMI 2022
Country/Territory	India
City	Bangalore
Period	7/11/22 → 11/11/22
Internet address	https://icmi.acm.org/2022/

Access to Document

10.1145/3536220.3563685

Cite this

Narayana, S., Subramanian, R., Radwan, I., & Goecke, R. (2022). To Improve Is to Change: Towards Improving Mood Prediction by Learning Changes in Emotion. In R. Tumuluri, N. Sebe, G. Pingali, & D. B. Jayagopi (Eds.), ICMI 2022 - Companion Publication of the 2022 International Conference on Multimodal Interaction (pp. 36-41). (ACM International Conference Proceeding Series). Association for Computing Machinery (ACM). https://doi.org/10.1145/3536220.3563685

Narayana, Soujanya ; Subramanian, Ramanathan ; Radwan, Ibrahim et al. / To Improve Is to Change: Towards Improving Mood Prediction by Learning Changes in Emotion. ICMI 2022 - Companion Publication of the 2022 International Conference on Multimodal Interaction. editor / Raj Tumuluri ; Nicu Sebe ; Gopal Pingali ; Dinesh Babu Jayagopi. New York : Association for Computing Machinery (ACM), 2022. pp. 36-41 (ACM International Conference Proceeding Series).

@inproceedings{a65433c1ebfc4c72bcf8df690daa4584,

title = "To Improve Is to Change: Towards Improving Mood Prediction by Learning Changes in Emotion",

abstract = "Although the terms mood and emotion are closely related and often used interchangeably, they are distinguished based on their duration, intensity and attribution. To date, hardly any computational models have (a) examined mood recognition, and (b) modelled the interplay between mood and emotional state in their analysis. In this paper, as a first step towards mood prediction, we propose a framework that utilises both dominant emotion (or mood) labels, and emotional change labels on the AFEW-VA database. Experiments evaluating unimodal (trained only using mood labels) and multimodal (trained with both mood and emotion change labels) convolutional neural networks confirm that incorporating emotional change information in the network training process can significantly improve the mood prediction performance, thus highlighting the importance of modelling emotion and mood simultaneously for improved performance in affective state recognition.",

keywords = "Mood, Emotion, Convolution neural network, Unimodal, Multimodal",

author = "Soujanya Narayana and Ramanathan Subramanian and Ibrahim Radwan and Roland Goecke",

note = "Funding Information: This research was supported partially by the Australian Government through the Australian Research Council{\textquoteright}s Discovery Projects funding scheme (project DP190101294). Publisher Copyright: {\textcopyright} 2022 ACM.; 24th ACM International Conference on Multimodal Interaction, ICMI 2022, ICMI 2022 ; Conference date: 07-11-2022 Through 11-11-2022",

year = "2022",

month = nov,

day = "7",

doi = "10.1145/3536220.3563685",

language = "English",

isbn = "9781450393898",

series = "ACM International Conference Proceeding Series",

publisher = "Association for Computing Machinery (ACM)",

pages = "36--41",

editor = "Raj Tumuluri and Nicu Sebe and Gopal Pingali and Jayagopi, {Dinesh Babu}",

booktitle = "ICMI 2022 - Companion Publication of the 2022 International Conference on Multimodal Interaction",

address = "United States",

url = "https://icmi.acm.org/2022/",

}

Narayana, S, Subramanian, R , Radwan, I & Goecke, R 2022, To Improve Is to Change: Towards Improving Mood Prediction by Learning Changes in Emotion. in R Tumuluri, N Sebe, G Pingali & DB Jayagopi (eds), ICMI 2022 - Companion Publication of the 2022 International Conference on Multimodal Interaction. ACM International Conference Proceeding Series, Association for Computing Machinery (ACM), New York, pp. 36-41, 24th ACM International Conference on Multimodal Interaction, ICMI 2022, Bangalore, India, 7/11/22. https://doi.org/10.1145/3536220.3563685

To Improve Is to Change: Towards Improving Mood Prediction by Learning Changes in Emotion. / Narayana, Soujanya; Subramanian, Ramanathan ; Radwan, Ibrahim et al.
ICMI 2022 - Companion Publication of the 2022 International Conference on Multimodal Interaction. ed. / Raj Tumuluri; Nicu Sebe; Gopal Pingali; Dinesh Babu Jayagopi. New York: Association for Computing Machinery (ACM), 2022. p. 36-41 (ACM International Conference Proceeding Series).

Research output: A Conference proceeding or a Chapter in Book › Conference contribution › peer-review

TY - GEN

T1 - To Improve Is to Change: Towards Improving Mood Prediction by Learning Changes in Emotion

AU - Narayana, Soujanya

AU - Subramanian, Ramanathan

AU - Radwan, Ibrahim

AU - Goecke, Roland

N1 - Funding Information: This research was supported partially by the Australian Government through the Australian Research Council’s Discovery Projects funding scheme (project DP190101294). Publisher Copyright: © 2022 ACM.

PY - 2022/11/7

Y1 - 2022/11/7

N2 - Although the terms mood and emotion are closely related and often used interchangeably, they are distinguished based on their duration, intensity and attribution. To date, hardly any computational models have (a) examined mood recognition, and (b) modelled the interplay between mood and emotional state in their analysis. In this paper, as a first step towards mood prediction, we propose a framework that utilises both dominant emotion (or mood) labels, and emotional change labels on the AFEW-VA database. Experiments evaluating unimodal (trained only using mood labels) and multimodal (trained with both mood and emotion change labels) convolutional neural networks confirm that incorporating emotional change information in the network training process can significantly improve the mood prediction performance, thus highlighting the importance of modelling emotion and mood simultaneously for improved performance in affective state recognition.

AB - Although the terms mood and emotion are closely related and often used interchangeably, they are distinguished based on their duration, intensity and attribution. To date, hardly any computational models have (a) examined mood recognition, and (b) modelled the interplay between mood and emotional state in their analysis. In this paper, as a first step towards mood prediction, we propose a framework that utilises both dominant emotion (or mood) labels, and emotional change labels on the AFEW-VA database. Experiments evaluating unimodal (trained only using mood labels) and multimodal (trained with both mood and emotion change labels) convolutional neural networks confirm that incorporating emotional change information in the network training process can significantly improve the mood prediction performance, thus highlighting the importance of modelling emotion and mood simultaneously for improved performance in affective state recognition.

KW - Mood

KW - Emotion

KW - Convolution neural network

KW - Unimodal

KW - Multimodal

UR - http://www.scopus.com/inward/record.url?scp=85142345161&partnerID=8YFLogxK

U2 - 10.1145/3536220.3563685

DO - 10.1145/3536220.3563685

M3 - Conference contribution

SN - 9781450393898

T3 - ACM International Conference Proceeding Series

SP - 36

EP - 41

BT - ICMI 2022 - Companion Publication of the 2022 International Conference on Multimodal Interaction

A2 - Tumuluri, Raj

A2 - Sebe, Nicu

A2 - Pingali, Gopal

A2 - Jayagopi, Dinesh Babu

PB - Association for Computing Machinery (ACM)

CY - New York

T2 - 24th ACM International Conference on Multimodal Interaction, ICMI 2022

Y2 - 7 November 2022 through 11 November 2022

ER -

Narayana S, Subramanian R , Radwan I, Goecke R. To Improve Is to Change: Towards Improving Mood Prediction by Learning Changes in Emotion. In Tumuluri R, Sebe N, Pingali G, Jayagopi DB, editors, ICMI 2022 - Companion Publication of the 2022 International Conference on Multimodal Interaction. New York: Association for Computing Machinery (ACM). 2022. p. 36-41. (ACM International Conference Proceeding Series). doi: 10.1145/3536220.3563685

To Improve Is to Change: Towards Improving Mood Prediction by Learning Changes in Emotion

Abstract

Publication series

Conference

Access to Document

Other files and links

Fingerprint

Cite this