TY - GEN
T1 - A Weakly Supervised Approach to Emotion-change Prediction and Improved Mood Inference
AU - Narayana, Soujanya
AU - Radwan, Ibrahim
AU - Parameshwara, Ravikiran
AU - Abbasnejad, Iman
AU - Asthana, Akshay
AU - Subramanian, Ramanathan
AU - Goecke, Roland
N1 - Funding Information:
This research is partially funded by the Australian Government through the Australian Research Council’s Discovery Projects funding scheme (project DP190101294). *Corresponding author: [email protected]
Publisher Copyright:
© 2023 IEEE.
PY - 2023/9/10
Y1 - 2023/9/10
N2 - Whilst a majority of affective computing research focuses on inferring emotions, examining mood or understanding the mood-emotion interplay has received significantly less attention. Building on prior work, we (a) deduce and incorporate emotion-change (Δ) information for inferring mood, without resorting to annotated labels, and (b) attempt mood prediction for long duration video clips, in alignment with the characterisation of mood. We generate the emotion-change (Δ) labels via metric learning from a pre-trained Siamese Network, and use these in addition to mood labels for mood classification. Experiments evaluating unimodal (training only using mood labels) vs muttimodat (training using mood plus Δ labels) models show that mood prediction benefits from the incorporation of emotion-change information, emphasising the importance of modelling the moodemotion interplay for effective mood inference.
AB - Whilst a majority of affective computing research focuses on inferring emotions, examining mood or understanding the mood-emotion interplay has received significantly less attention. Building on prior work, we (a) deduce and incorporate emotion-change (Δ) information for inferring mood, without resorting to annotated labels, and (b) attempt mood prediction for long duration video clips, in alignment with the characterisation of mood. We generate the emotion-change (Δ) labels via metric learning from a pre-trained Siamese Network, and use these in addition to mood labels for mood classification. Experiments evaluating unimodal (training only using mood labels) vs muttimodat (training using mood plus Δ labels) models show that mood prediction benefits from the incorporation of emotion-change information, emphasising the importance of modelling the moodemotion interplay for effective mood inference.
KW - Contrastive Loss
KW - Emotion change
KW - Mood inference
KW - Multimodal
KW - Siamese network
KW - Teacher-student network
KW - Unimodal
UR - http://www.scopus.com/inward/record.url?scp=85169168652&partnerID=8YFLogxK
UR - https://ieeexplore.ieee.org/document/10388146
UR - https://ieeexplore.ieee.org/xpl/conhome/10388070/proceeding
UR - http://asoca.ewi.tudelft.nl/
U2 - 10.1109/ACII59096.2023.10388146
DO - 10.1109/ACII59096.2023.10388146
M3 - Conference contribution
AN - SCOPUS:85169168652
SN - 9798350327441
T3 - 2023 11th International Conference on Affective Computing and Intelligent Interaction, ACII 2023
SP - 1
EP - 8
BT - 2023 11th International Conference on Affective Computing and Intelligent Interaction, ACII 2023
PB - IEEE, Institute of Electrical and Electronics Engineers
CY - United States
T2 - 11th International Conference on Affective Computing and Intelligent Interaction, ACII 2023
Y2 - 10 September 2023 through 13 September 2023
ER -