An RGB-D based image set classification for robust face recognition from Kinect data

Munawar Hayat, Mohammed Bennamoun, Amar El-Sallam

Research output: Contribution to journalArticle

12 Citations (Scopus)

Abstract

The paper proposes a method for robust face recognition from low quality Kinect acquired images which have a wide range of variations in head pose, illumination, facial expressions, sunglass disguise and occlusions by hand. Multiple Kinect images of a person are considered as an image set and face recognition from these images is formulated as an RGB–D image set classification problem. The Kinect acquired raw depth data is used for pose estimation and an automatic cropping of the face region. Based upon the estimated poses, the face images of a set are divided into multiple image subsets. An efficient block based covariance matrix representation is proposed to model images in an image subset on Riemannian manifold (Lie group). For classification, SVM models are separately learnt for each image subset on the Lie group of Riemannian manifold and a fusion strategy is introduced to combine results from all image subsets. The proposed technique has been evaluated on a combination of three large data sets containing over 35,000 RGB–D images under challenging conditions. The proposed RGB–D based image set classification incurs low computational cost and achieves an identification rate as high as 99.5%
Original languageEnglish
Pages (from-to)889-900
Number of pages12
JournalNeurocomputing: an international journal
Volume171
DOIs
Publication statusPublished - 2016
Externally publishedYes

Fingerprint

Face recognition
Lie groups
Set theory
Facial Expression
Covariance matrix
Lighting
Fusion reactions
Hand
Head
Costs and Cost Analysis
Facial Recognition
Costs

Cite this

@article{486afbedbf2d4d05ae29b21345f086d7,
title = "An RGB-D based image set classification for robust face recognition from Kinect data",
abstract = "The paper proposes a method for robust face recognition from low quality Kinect acquired images which have a wide range of variations in head pose, illumination, facial expressions, sunglass disguise and occlusions by hand. Multiple Kinect images of a person are considered as an image set and face recognition from these images is formulated as an RGB–D image set classification problem. The Kinect acquired raw depth data is used for pose estimation and an automatic cropping of the face region. Based upon the estimated poses, the face images of a set are divided into multiple image subsets. An efficient block based covariance matrix representation is proposed to model images in an image subset on Riemannian manifold (Lie group). For classification, SVM models are separately learnt for each image subset on the Lie group of Riemannian manifold and a fusion strategy is introduced to combine results from all image subsets. The proposed technique has been evaluated on a combination of three large data sets containing over 35,000 RGB–D images under challenging conditions. The proposed RGB–D based image set classification incurs low computational cost and achieves an identification rate as high as 99.5{\%}",
keywords = "face-recognition, computer-vision, image-processing",
author = "Munawar Hayat and Mohammed Bennamoun and Amar El-Sallam",
year = "2016",
doi = "10.1016/j.neucom.2015.07.027",
language = "English",
volume = "171",
pages = "889--900",
journal = "Neurocomputing: an international journal",
issn = "0925-2312",
publisher = "Elsevier",

}

An RGB-D based image set classification for robust face recognition from Kinect data. / Hayat, Munawar; Bennamoun, Mohammed; El-Sallam, Amar.

In: Neurocomputing: an international journal, Vol. 171, 2016, p. 889-900.

Research output: Contribution to journalArticle

TY - JOUR

T1 - An RGB-D based image set classification for robust face recognition from Kinect data

AU - Hayat, Munawar

AU - Bennamoun, Mohammed

AU - El-Sallam, Amar

PY - 2016

Y1 - 2016

N2 - The paper proposes a method for robust face recognition from low quality Kinect acquired images which have a wide range of variations in head pose, illumination, facial expressions, sunglass disguise and occlusions by hand. Multiple Kinect images of a person are considered as an image set and face recognition from these images is formulated as an RGB–D image set classification problem. The Kinect acquired raw depth data is used for pose estimation and an automatic cropping of the face region. Based upon the estimated poses, the face images of a set are divided into multiple image subsets. An efficient block based covariance matrix representation is proposed to model images in an image subset on Riemannian manifold (Lie group). For classification, SVM models are separately learnt for each image subset on the Lie group of Riemannian manifold and a fusion strategy is introduced to combine results from all image subsets. The proposed technique has been evaluated on a combination of three large data sets containing over 35,000 RGB–D images under challenging conditions. The proposed RGB–D based image set classification incurs low computational cost and achieves an identification rate as high as 99.5%

AB - The paper proposes a method for robust face recognition from low quality Kinect acquired images which have a wide range of variations in head pose, illumination, facial expressions, sunglass disguise and occlusions by hand. Multiple Kinect images of a person are considered as an image set and face recognition from these images is formulated as an RGB–D image set classification problem. The Kinect acquired raw depth data is used for pose estimation and an automatic cropping of the face region. Based upon the estimated poses, the face images of a set are divided into multiple image subsets. An efficient block based covariance matrix representation is proposed to model images in an image subset on Riemannian manifold (Lie group). For classification, SVM models are separately learnt for each image subset on the Lie group of Riemannian manifold and a fusion strategy is introduced to combine results from all image subsets. The proposed technique has been evaluated on a combination of three large data sets containing over 35,000 RGB–D images under challenging conditions. The proposed RGB–D based image set classification incurs low computational cost and achieves an identification rate as high as 99.5%

KW - face-recognition

KW - computer-vision

KW - image-processing

U2 - 10.1016/j.neucom.2015.07.027

DO - 10.1016/j.neucom.2015.07.027

M3 - Article

VL - 171

SP - 889

EP - 900

JO - Neurocomputing: an international journal

JF - Neurocomputing: an international journal

SN - 0925-2312

ER -