Making computers look the way we look: Exploiting visual attention for image understanding

Harish Katti, Ramanathan Subramanian, Mohan Kankanhalli, Nicu Sebe, Tat Seng Chua, Kalpathi R. Ramakrishnan

Research output: A Conference proceeding or a Chapter in BookConference contributionpeer-review

12 Citations (Scopus)

Abstract

Human Visual attention (HVA) is an important strategy to focus on specific information while observing and understanding visual stimuli. HVA involves making a series of fixations on select locations while performing tasks such as object recognition, scene understanding, etc. We present one of the first works that combines fixation information with automated concept detectors to (i) infer abstract image semantics, and (ii) enhance performance of object detectors. We develop visual attention-based models that sample fixation distributions and fixation transition distributions in regions-of-interest (ROI) to infer abstract semantics such as expressive faces and interactions (such as look, read, etc.). We also exploit eye-gaze information to deduce possible locations and scale of salient concepts and aid state-of-art detectors. A 18% performance increase with over 80% reduction in computational time for a state-of-art object detector [4].

Original languageEnglish
Title of host publicationMM'10 - Proceedings of the ACM Multimedia 2010 International Conference
EditorsAlberto del Bimbo, Shih-Fu Chang, Arnold Smeulders
Place of PublicationUnited States
PublisherAssociation for Computing Machinery (ACM)
Pages667-670
Number of pages4
ISBN (Print)9781605589336
DOIs
Publication statusPublished - 2010
Externally publishedYes

Publication series

NameMM'10 - Proceedings of the ACM Multimedia 2010 International Conference

Fingerprint

Dive into the research topics of 'Making computers look the way we look: Exploiting visual attention for image understanding'. Together they form a unique fingerprint.

Cite this