Joint Registration and Representation Learning for Unconstrained Face Identification

Munawar HAYAT, Salman Khan, Naoufel Werghi, Roland GOECKE

Research output: A Conference proceeding or a Chapter in BookConference contribution

11 Citations (Scopus)

Abstract

Recent advances in deep learning have resulted in human-level performances on popular unconstrained face datasets including Labeled Faces in the Wild and YouTube Faces. To further advance research, IJB-A benchmark was recently introduced with more challenges especially in the form of extreme head poses. Registration of such faces is quite demanding and often requires laborious procedures like facial landmark localization. In this paper, we propose a Convolutional Neural Networks based data-driven approach which learns to simultaneously register and represent faces. We validate the proposed scheme on template based unconstrained face identification. Here, a template contains multiple media in the form of images and video frames. Unlike existing methods which synthesize all template media information at feature level, we propose to keep the template media intact. Instead, we represent gallery templates by their trained one-vs-rest discriminative models and then employ a Bayesian strategy which optimally fuses decisions of all medias in a query template. We demonstrate the efficacy of the proposed scheme on IJB-A, YouTube Celebrities and COX datasets where our approach achieves significant relative performance boosts of 3.6%, 21.6% and 12.8% respectively.
Original languageEnglish
Title of host publicationProceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR’17)
Place of PublicationHonolulu, Hawaii, USA
PublisherIEEE
Pages1551-1560
Number of pages10
ISBN (Electronic)9781538604571
ISBN (Print)9781538604588
DOIs
Publication statusPublished - 6 Nov 2017
Event2017 IEEE Conference on Computer Vision and Pattern Recognition - Hawaii Convention Center, Honolulu, United States
Duration: 21 Jul 201726 Jul 2017
http://cvpr2017.thecvf.com/

Publication series

Name30TH IEEE CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION (CVPR 2017)
ISSN (Print)1063-6919

Conference

Conference2017 IEEE Conference on Computer Vision and Pattern Recognition
Abbreviated titleCVPR 2017
CountryUnited States
CityHonolulu
Period21/07/1726/07/17
Internet address

Fingerprint

Electric fuses
Neural networks
Deep learning

Cite this

HAYAT, M., Khan, S., Werghi, N., & GOECKE, R. (2017). Joint Registration and Representation Learning for Unconstrained Face Identification. In Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR’17) (pp. 1551-1560). (30TH IEEE CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION (CVPR 2017)). Honolulu, Hawaii, USA: IEEE. https://doi.org/10.1109/CVPR.2017.169
HAYAT, Munawar ; Khan, Salman ; Werghi, Naoufel ; GOECKE, Roland. / Joint Registration and Representation Learning for Unconstrained Face Identification. Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR’17). Honolulu, Hawaii, USA : IEEE, 2017. pp. 1551-1560 (30TH IEEE CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION (CVPR 2017)).
@inproceedings{fe6e12caf6174e1681402659400b9121,
title = "Joint Registration and Representation Learning for Unconstrained Face Identification",
abstract = "Recent advances in deep learning have resulted in human-level performances on popular unconstrained face datasets including Labeled Faces in the Wild and YouTube Faces. To further advance research, IJB-A benchmark was recently introduced with more challenges especially in the form of extreme head poses. Registration of such faces is quite demanding and often requires laborious procedures like facial landmark localization. In this paper, we propose a Convolutional Neural Networks based data-driven approach which learns to simultaneously register and represent faces. We validate the proposed scheme on template based unconstrained face identification. Here, a template contains multiple media in the form of images and video frames. Unlike existing methods which synthesize all template media information at feature level, we propose to keep the template media intact. Instead, we represent gallery templates by their trained one-vs-rest discriminative models and then employ a Bayesian strategy which optimally fuses decisions of all medias in a query template. We demonstrate the efficacy of the proposed scheme on IJB-A, YouTube Celebrities and COX datasets where our approach achieves significant relative performance boosts of 3.6{\%}, 21.6{\%} and 12.8{\%} respectively.",
keywords = "Face Recognition, Face Identification, Registration",
author = "Munawar HAYAT and Salman Khan and Naoufel Werghi and Roland GOECKE",
year = "2017",
month = "11",
day = "6",
doi = "10.1109/CVPR.2017.169",
language = "English",
isbn = "9781538604588",
series = "30TH IEEE CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION (CVPR 2017)",
publisher = "IEEE",
pages = "1551--1560",
booktitle = "Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR’17)",

}

HAYAT, M, Khan, S, Werghi, N & GOECKE, R 2017, Joint Registration and Representation Learning for Unconstrained Face Identification. in Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR’17). 30TH IEEE CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION (CVPR 2017), IEEE, Honolulu, Hawaii, USA, pp. 1551-1560, 2017 IEEE Conference on Computer Vision and Pattern Recognition, Honolulu, United States, 21/07/17. https://doi.org/10.1109/CVPR.2017.169

Joint Registration and Representation Learning for Unconstrained Face Identification. / HAYAT, Munawar; Khan, Salman; Werghi, Naoufel; GOECKE, Roland.

Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR’17). Honolulu, Hawaii, USA : IEEE, 2017. p. 1551-1560 (30TH IEEE CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION (CVPR 2017)).

Research output: A Conference proceeding or a Chapter in BookConference contribution

TY - GEN

T1 - Joint Registration and Representation Learning for Unconstrained Face Identification

AU - HAYAT, Munawar

AU - Khan, Salman

AU - Werghi, Naoufel

AU - GOECKE, Roland

PY - 2017/11/6

Y1 - 2017/11/6

N2 - Recent advances in deep learning have resulted in human-level performances on popular unconstrained face datasets including Labeled Faces in the Wild and YouTube Faces. To further advance research, IJB-A benchmark was recently introduced with more challenges especially in the form of extreme head poses. Registration of such faces is quite demanding and often requires laborious procedures like facial landmark localization. In this paper, we propose a Convolutional Neural Networks based data-driven approach which learns to simultaneously register and represent faces. We validate the proposed scheme on template based unconstrained face identification. Here, a template contains multiple media in the form of images and video frames. Unlike existing methods which synthesize all template media information at feature level, we propose to keep the template media intact. Instead, we represent gallery templates by their trained one-vs-rest discriminative models and then employ a Bayesian strategy which optimally fuses decisions of all medias in a query template. We demonstrate the efficacy of the proposed scheme on IJB-A, YouTube Celebrities and COX datasets where our approach achieves significant relative performance boosts of 3.6%, 21.6% and 12.8% respectively.

AB - Recent advances in deep learning have resulted in human-level performances on popular unconstrained face datasets including Labeled Faces in the Wild and YouTube Faces. To further advance research, IJB-A benchmark was recently introduced with more challenges especially in the form of extreme head poses. Registration of such faces is quite demanding and often requires laborious procedures like facial landmark localization. In this paper, we propose a Convolutional Neural Networks based data-driven approach which learns to simultaneously register and represent faces. We validate the proposed scheme on template based unconstrained face identification. Here, a template contains multiple media in the form of images and video frames. Unlike existing methods which synthesize all template media information at feature level, we propose to keep the template media intact. Instead, we represent gallery templates by their trained one-vs-rest discriminative models and then employ a Bayesian strategy which optimally fuses decisions of all medias in a query template. We demonstrate the efficacy of the proposed scheme on IJB-A, YouTube Celebrities and COX datasets where our approach achieves significant relative performance boosts of 3.6%, 21.6% and 12.8% respectively.

KW - Face Recognition

KW - Face Identification

KW - Registration

UR - http://www.scopus.com/inward/record.url?scp=85041928374&partnerID=8YFLogxK

U2 - 10.1109/CVPR.2017.169

DO - 10.1109/CVPR.2017.169

M3 - Conference contribution

SN - 9781538604588

T3 - 30TH IEEE CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION (CVPR 2017)

SP - 1551

EP - 1560

BT - Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR’17)

PB - IEEE

CY - Honolulu, Hawaii, USA

ER -

HAYAT M, Khan S, Werghi N, GOECKE R. Joint Registration and Representation Learning for Unconstrained Face Identification. In Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR’17). Honolulu, Hawaii, USA: IEEE. 2017. p. 1551-1560. (30TH IEEE CONFERENCE ON COMPUTER VISION AND PATTERN RECOGNITION (CVPR 2017)). https://doi.org/10.1109/CVPR.2017.169