MAGIC-TBR: Multiview Attention Fusion for Transformer-based Bodily Behavior Recognition in Group Settings

Surbhi Madan, Rishabh Jain, Gulshan Sharma, Ramanathan Subramanian, Abhinav Dhall

Research output: A Conference proceeding or a Chapter in BookConference contributionpeer-review

Abstract

Bodily behavioral language is an important social cue, and its automated analysis helps in enhancing the understanding of artificial intelligence systems. Furthermore, behavioral language cues are essential for active engagement in social agent-based user interactions. Despite the progress made in computer vision for tasks like head and body pose estimation, there is still a need to explore the detection of finer behaviors such as gesturing, grooming, or fumbling. This paper proposes a multiview attention fusion method named MAGIC-TBR that combines features extracted from videos and their corresponding Discrete Cosine Transform coefficients via a transformer-based approach. The experiments are conducted on the BBSI dataset and the results demonstrate the effectiveness of the proposed feature fusion with multiview attention. The code is available at: https://github.com/surbhimadan92/MAGIC-TBR
Original languageEnglish
Title of host publicationProceedings of the 31st ACM International Conference on Multimedia
Subtitle of host publicationMM23
EditorsAbdulmotaleb El Saddik, Tao Mei , Rita Cucchiara, Marco Bertini, Diana Patricia, Pradeep K. Atrey, M. Shamim Hossain, Mukesh K. Saini, Ming-Ching Chang
Place of PublicationUnited States
PublisherAssociation for Computing Machinery (ACM)
Pages9526-9530
Number of pages5
ISBN (Electronic)9798400701085
ISBN (Print)9798400701085
DOIs
Publication statusPublished - 26 Oct 2023
Event31st ACM International Conference on Multimedia 2023 - Westin Ottawa Hotel, Ottawa, Canada
Duration: 29 Oct 20233 Nov 2023
https://www.acmmm2023.org/

Publication series

NameMM 2023 - Proceedings of the 31st ACM International Conference on Multimedia

Conference

Conference31st ACM International Conference on Multimedia 2023
Abbreviated titleACMMM '23
Country/TerritoryCanada
CityOttawa
Period29/10/233/11/23
Internet address

Fingerprint

Dive into the research topics of 'MAGIC-TBR: Multiview Attention Fusion for Transformer-based Bodily Behavior Recognition in Group Settings'. Together they form a unique fingerprint.

Cite this