Examining Subject-Dependent and Subject-Independent Human Affect Inference from Limited Video Data

Ravikiran Parameshwara, Ibrahim Radwan, Ramanathan Subramanian, Roland Goecke

Research output: A Conference proceeding or a Chapter in BookConference contributionpeer-review

2 Citations (Scopus)

Abstract

Continuous human affect estimation from video data entails modelling the dynamic emotional state from a sequence of facial images. Though multiple affective video databases exist, they are limited in terms of data and dynamic annotations, as assigning continuous affective labels to video data is subjective, onerous and tedious. While studies have established the existence of signature facial expressions corresponding to the basic categorical emotions, individual differences in emoting facial expressions nevertheless exist; factoring out these idiosyncrasies is critical for effective emotion inference. This work explores continuous human affect recognition using AFEW-VA, an 'in-the-wild' video dataset with limited data, employing subject-independent (SI) and subject-dependent (SD) settings. The SI setting involves the use of training and test sets with mutually exclusive subjects, while training and test samples corresponding to the same subject can occur in the SD setting. A novel, dynamically-weighted loss function is employed with a Convolutional Neural Network (CNN)-Long Short- Term Memory (LSTM) architecture to optimise dynamic affect prediction. Superior prediction is achieved in the SD setting, as compared to the SI counterpart.

Original languageEnglish
Title of host publication2023 IEEE 17th International Conference on Automatic Face and Gesture Recognition, FG 2023
PublisherIEEE, Institute of Electrical and Electronics Engineers
Pages1-6
Number of pages6
ISBN (Electronic)9798350345445
DOIs
Publication statusPublished - 5 Jan 2023
Event17th IEEE International Conference on Automatic Face and Gesture Recognition - Waikoloa Beach, United States
Duration: 5 Jan 20238 Jan 2023

Publication series

Name2023 IEEE 17th International Conference on Automatic Face and Gesture Recognition, FG 2023

Conference

Conference17th IEEE International Conference on Automatic Face and Gesture Recognition
Abbreviated titleFG 2023
Country/TerritoryUnited States
CityWaikoloa Beach
Period5/01/238/01/23

Fingerprint

Dive into the research topics of 'Examining Subject-Dependent and Subject-Independent Human Affect Inference from Limited Video Data'. Together they form a unique fingerprint.

Cite this