Continuous authentication using EEG and face images for trusted autonomous systems

Human identity is a prerequisite for trust assurance and assessment, which is essential for effective human-machine interaction in trusted autonomous systems. Unlike conventional authentication methods which do not require users to re-authenticate themselves for sustained access, continuous authentication affirms human identity in real-time, therefore is a solution for continued access monitoring in trusted autonomous systems. Robust continuous authentication needs robust multi-modal data sources. In this paper, we design a multi-modal biometrics system that continuously verifies the presence of a logged-in user. Two types of biometric data are used, face images and Electroencephalography (EEG) signals. Information from individual modalities is fused at matching score level. For face modality, matching scores are calculated by distances between eigenface coefficients. While for EEG signals, an event-related potential (ERP) modality is established by a simple ERP elicitation protocol and calculation of cross-correlation similarities. Scores from the two modalities are normalized and fused using three schemes, namely the sum-score, max-score and min-score scheme. The experiments reveal that individual variations found in the ERPs are detectable and can be used for continuous authentication. This is an interesting finding which indicates that the ERP biometrics are feasible for user authentication and worthy of further research. Results also show that combining ERP biometric with face biometric using sum-score scheme outperforms each modality in isolation. This piece of finding indicates the potential of integrating ERP into multimodal authentication systems.