Head Matters: Explainable Human-centered Trait Prediction from Head Motion Dynamics

Surbhi Madan, Monika Gahalawat, Tanaya Guha, Ramanathan Subramanian

Research output: A Conference proceeding or a Chapter in BookConference contributionpeer-review

Abstract

We demonstrate the utility of elementary head-motion units termed kinemes for behavioral analytics to predict personality and interview traits. Transforming head-motion patterns into a sequence of kinemes facilitates discovery of latent temporal signatures characterizing the targeted traits, thereby enabling both efficient and explainable trait prediction. Utilizing Kinemes and Facial Action Coding System (FACS) features to predict (a) OCEAN personality traits on the First Impressions Candidate Screening videos, and (b) Interview traits on the MIT dataset, we note that: (1) A Long-Short Term Memory (LSTM) network trained with kineme sequences performs better than or similar to a Convolutional Neural Network (CNN) trained with facial images; (2) Accurate predictions and explanations are achieved on combining FACS action units (AUs) with kinemes, and (3) Prediction performance is affected by the time-length over which head and facial movements are observed.

Original languageEnglish
Title of host publicationICMI 2021 - Proceedings of the 2021 International Conference on Multimodal Interaction
EditorsZakia Hammal, Carlos Busso, Catherine Pelachaud, Sharon Oviatt, Albert Ali Salah, Guoying Zhao
Place of PublicationUnited States
PublisherAssociation for Computing Machinery (ACM)
Pages435-443
Number of pages9
ISBN (Electronic)9781450384810
DOIs
Publication statusPublished - 18 Oct 2021
Event23rd ACM International Conference on Multimodal Interaction, ICMI 2021 - Virtual, Online, Canada
Duration: 18 Oct 202122 Oct 2021

Publication series

NameICMI 2021 - Proceedings of the 2021 International Conference on Multimodal Interaction

Conference

Conference23rd ACM International Conference on Multimodal Interaction, ICMI 2021
Country/TerritoryCanada
CityVirtual, Online
Period18/10/2122/10/21

Fingerprint

Dive into the research topics of 'Head Matters: Explainable Human-centered Trait Prediction from Head Motion Dynamics'. Together they form a unique fingerprint.

Cite this