The eyes know it: FakeET- An Eye-tracking Database to Understand Deepfake Perception

Parul Gupta, Komal Chugh, Abhinav Dhall, Ramanathan Subramanian

Research output: A Conference proceeding or a Chapter in BookConference contributionpeer-review

Abstract

We present FakeET - an eye-tracking database to understand human visual perception of deepfake videos. Given that the principal purpose of deepfakes is to deceive human observers, FakeET is designed to understand and evaluate the ability of viewers to detect synthetic video artifacts. FakeET contains viewing patterns compiled from 40 users via the Tobii desktop eye-tracker for 811 videos from the Google Deepfake dataset, with a minimum of two viewings per video. Additionally, EEG responses acquired via the Emotiv sensor are also available. The compiled data confirms (a) distinct eye movement characteristics for real vs fake videos; (b) utility of the eye-track saliency maps for spatial forgery localization and detection, and (c) Error Related Negativity (ERN) triggers in the EEG responses, and the ability of the raw EEG signal to distinguish between real and fake videos.

Original languageEnglish
Title of host publicationICMI 2020 - Proceedings of the 2020 International Conference on Multimodal Interaction
EditorsKhiet Truong, Dirk Heylen
Place of PublicationUnited States
PublisherAssociation for Computing Machinery (ACM)
Pages519-527
Number of pages9
ISBN (Electronic)9781450375818
DOIs
Publication statusPublished - 21 Oct 2020
Externally publishedYes
Event22nd ACM International Conference on Multimodal Interaction, ICMI 2020 - Virtual, Online, Netherlands
Duration: 25 Oct 202029 Oct 2020

Publication series

NameICMI 2020 - Proceedings of the 2020 International Conference on Multimodal Interaction

Conference

Conference22nd ACM International Conference on Multimodal Interaction, ICMI 2020
CountryNetherlands
CityVirtual, Online
Period25/10/2029/10/20

Fingerprint Dive into the research topics of 'The eyes know it: FakeET- An Eye-tracking Database to Understand Deepfake Perception'. Together they form a unique fingerprint.

Cite this