Deep reconstruction models for image set classification

Munawar Hayat, Mohammed Bennamoun, Senjian An

Research output: Contribution to journalArticle

94 Citations (Scopus)
4 Downloads (Pure)

Abstract

Image set classification finds its applications in a number of real-life scenarios such as classification from surveillance videos, multi-view camera networks and personal albums. Compared with single image based classification, it offers more promises and has therefore attracted significant research attention in recent years. Unlike many existing methods which assume images of a set to lie on a certain geometric surface, this paper introduces a deep learning framework which makes no such prior assumptions and can automatically discover the underlying geometric structure. Specifically, a Template Deep Reconstruction Model (TDRM) is defined whose parameters are initialized by performing unsupervised pre-training in a layer-wise fashion using Gaussian Restricted Boltzmann Machines (GRBMs). The initialized TDRM is then separately trained for images of each class and class-specific DRMs are learnt. Based on the minimum reconstruction errors from the learnt class-specific models, three different voting strategies are devised for classification. Extensive experiments are performed to demonstrate the efficacy of the proposed framework for the tasks of face and object recognition from image sets. Experimental results show that the proposed method consistently outperforms the existing state of the art methods.

Original languageEnglish
Article number6888522
Pages (from-to)713-727
Number of pages15
JournalIEEE Transactions on Pattern Analysis and Machine Intelligence
Volume37
Issue number4
DOIs
Publication statusPublished - 2015
Externally publishedYes

Fingerprint

Template
Object recognition
Boltzmann Machine
Face recognition
Model
Video Surveillance
Object Recognition
Geometric Structure
Voting
Face Recognition
Cameras
Efficacy
Camera
Scenarios
Experimental Results
Experiments
Demonstrate
Experiment
Class
Framework

Cite this

@article{41a6d2937f0646f6b558d0bfa26ed7cd,
title = "Deep reconstruction models for image set classification",
abstract = "Image set classification finds its applications in a number of real-life scenarios such as classification from surveillance videos, multi-view camera networks and personal albums. Compared with single image based classification, it offers more promises and has therefore attracted significant research attention in recent years. Unlike many existing methods which assume images of a set to lie on a certain geometric surface, this paper introduces a deep learning framework which makes no such prior assumptions and can automatically discover the underlying geometric structure. Specifically, a Template Deep Reconstruction Model (TDRM) is defined whose parameters are initialized by performing unsupervised pre-training in a layer-wise fashion using Gaussian Restricted Boltzmann Machines (GRBMs). The initialized TDRM is then separately trained for images of each class and class-specific DRMs are learnt. Based on the minimum reconstruction errors from the learnt class-specific models, three different voting strategies are devised for classification. Extensive experiments are performed to demonstrate the efficacy of the proposed framework for the tasks of face and object recognition from image sets. Experimental results show that the proposed method consistently outperforms the existing state of the art methods.",
keywords = "face-recognition, computer-vision, image-processing, Image set classification, auto-encoders, object recognition, deep learning, video based face recognition",
author = "Munawar Hayat and Mohammed Bennamoun and Senjian An",
year = "2015",
doi = "10.1109/tpami.2014.2353635",
language = "English",
volume = "37",
pages = "713--727",
journal = "IEEE Transactions on Pattern Analysis and Machine Intelligence",
issn = "0162-8828",
publisher = "IEEE, Institute of Electrical and Electronics Engineers",
number = "4",

}

Deep reconstruction models for image set classification. / Hayat, Munawar; Bennamoun, Mohammed; An, Senjian.

In: IEEE Transactions on Pattern Analysis and Machine Intelligence, Vol. 37, No. 4, 6888522, 2015, p. 713-727.

Research output: Contribution to journalArticle

TY - JOUR

T1 - Deep reconstruction models for image set classification

AU - Hayat, Munawar

AU - Bennamoun, Mohammed

AU - An, Senjian

PY - 2015

Y1 - 2015

N2 - Image set classification finds its applications in a number of real-life scenarios such as classification from surveillance videos, multi-view camera networks and personal albums. Compared with single image based classification, it offers more promises and has therefore attracted significant research attention in recent years. Unlike many existing methods which assume images of a set to lie on a certain geometric surface, this paper introduces a deep learning framework which makes no such prior assumptions and can automatically discover the underlying geometric structure. Specifically, a Template Deep Reconstruction Model (TDRM) is defined whose parameters are initialized by performing unsupervised pre-training in a layer-wise fashion using Gaussian Restricted Boltzmann Machines (GRBMs). The initialized TDRM is then separately trained for images of each class and class-specific DRMs are learnt. Based on the minimum reconstruction errors from the learnt class-specific models, three different voting strategies are devised for classification. Extensive experiments are performed to demonstrate the efficacy of the proposed framework for the tasks of face and object recognition from image sets. Experimental results show that the proposed method consistently outperforms the existing state of the art methods.

AB - Image set classification finds its applications in a number of real-life scenarios such as classification from surveillance videos, multi-view camera networks and personal albums. Compared with single image based classification, it offers more promises and has therefore attracted significant research attention in recent years. Unlike many existing methods which assume images of a set to lie on a certain geometric surface, this paper introduces a deep learning framework which makes no such prior assumptions and can automatically discover the underlying geometric structure. Specifically, a Template Deep Reconstruction Model (TDRM) is defined whose parameters are initialized by performing unsupervised pre-training in a layer-wise fashion using Gaussian Restricted Boltzmann Machines (GRBMs). The initialized TDRM is then separately trained for images of each class and class-specific DRMs are learnt. Based on the minimum reconstruction errors from the learnt class-specific models, three different voting strategies are devised for classification. Extensive experiments are performed to demonstrate the efficacy of the proposed framework for the tasks of face and object recognition from image sets. Experimental results show that the proposed method consistently outperforms the existing state of the art methods.

KW - face-recognition

KW - computer-vision

KW - image-processing

KW - Image set classification

KW - auto-encoders

KW - object recognition

KW - deep learning

KW - video based face recognition

UR - http://www.scopus.com/inward/record.url?scp=84924588005&partnerID=8YFLogxK

UR - http://www.mendeley.com/research/deep-reconstruction-models-image-set-classification

U2 - 10.1109/tpami.2014.2353635

DO - 10.1109/tpami.2014.2353635

M3 - Article

VL - 37

SP - 713

EP - 727

JO - IEEE Transactions on Pattern Analysis and Machine Intelligence

JF - IEEE Transactions on Pattern Analysis and Machine Intelligence

SN - 0162-8828

IS - 4

M1 - 6888522

ER -