Contractive rectifier networks for nonlinear maximum margin classification

Senjian An, Munawar Hayat, Salam Khan, Mohammed Bennamoun, Farid Boussaid, Ferdous Sohel

Research output: A Conference proceeding or a Chapter in BookConference contribution

3 Citations (Scopus)
2 Downloads (Pure)

Abstract

To find the optimal nonlinear separating boundary with maximum margin in the input data space, this paper proposes Contractive Rectifier Networks (CRNs), wherein the hidden-layer transformations are restricted to be contraction mappings. The contractive constraints ensure that the achieved separating margin in the input space is larger than or equal to the separating margin in the output layer. The training of the proposed CRNs is formulated as a linear support vector machine (SVM) in the output layer, combined with two or more contractive hidden layers. Effective algorithms have been proposed to address the optimization challenges arising from contraction constraints. Experimental results on MNIST, CIFAR-10, CIFAR-100 and MIT-67 datasets demonstrate that the proposed contractive rectifier networks consistently outperform their conventional unconstrained rectifier network counterparts
Original languageEnglish
Title of host publication2015 International Conference on Computer Vision, ICCV 2015
EditorsRuzena Bajcsy, Greg Hager, Yi Ma
Place of PublicationSantiago
PublisherIEEE, Institute of Electrical and Electronics Engineers
Pages2515-2523
Number of pages9
ISBN (Electronic)9781467383912
ISBN (Print)9781467383912
DOIs
Publication statusPublished - 2015
Externally publishedYes
Event2015 IEEE International Conference on Computer Vision - Santiago, Santiago, Chile
Duration: 11 Dec 201518 Dec 2015

Publication series

NameProceedings of the IEEE International Conference on Computer Vision
Volume2015 International Conference on Computer Vision, ICCV 2015
ISSN (Print)1550-5499

Conference

Conference2015 IEEE International Conference on Computer Vision
Abbreviated titleICCV 2015
CountryChile
CitySantiago
Period11/12/1518/12/15

Fingerprint

Support vector machines

Cite this

An, S., Hayat, M., Khan, S., Bennamoun, M., Boussaid, F., & Sohel, F. (2015). Contractive rectifier networks for nonlinear maximum margin classification. In R. Bajcsy, G. Hager, & Y. Ma (Eds.), 2015 International Conference on Computer Vision, ICCV 2015 (pp. 2515-2523). [7410646] (Proceedings of the IEEE International Conference on Computer Vision; Vol. 2015 International Conference on Computer Vision, ICCV 2015). Santiago: IEEE, Institute of Electrical and Electronics Engineers. https://doi.org/10.1109/iccv.2015.289
An, Senjian ; Hayat, Munawar ; Khan, Salam ; Bennamoun, Mohammed ; Boussaid, Farid ; Sohel, Ferdous. / Contractive rectifier networks for nonlinear maximum margin classification. 2015 International Conference on Computer Vision, ICCV 2015. editor / Ruzena Bajcsy ; Greg Hager ; Yi Ma. Santiago : IEEE, Institute of Electrical and Electronics Engineers, 2015. pp. 2515-2523 (Proceedings of the IEEE International Conference on Computer Vision).
@inproceedings{92011af6ce244542971433108034cf7d,
title = "Contractive rectifier networks for nonlinear maximum margin classification",
abstract = "To find the optimal nonlinear separating boundary with maximum margin in the input data space, this paper proposes Contractive Rectifier Networks (CRNs), wherein the hidden-layer transformations are restricted to be contraction mappings. The contractive constraints ensure that the achieved separating margin in the input space is larger than or equal to the separating margin in the output layer. The training of the proposed CRNs is formulated as a linear support vector machine (SVM) in the output layer, combined with two or more contractive hidden layers. Effective algorithms have been proposed to address the optimization challenges arising from contraction constraints. Experimental results on MNIST, CIFAR-10, CIFAR-100 and MIT-67 datasets demonstrate that the proposed contractive rectifier networks consistently outperform their conventional unconstrained rectifier network counterparts",
keywords = "Deep learning, Pattern recognition, Computer vision",
author = "Senjian An and Munawar Hayat and Salam Khan and Mohammed Bennamoun and Farid Boussaid and Ferdous Sohel",
year = "2015",
doi = "10.1109/iccv.2015.289",
language = "English",
isbn = "9781467383912",
series = "Proceedings of the IEEE International Conference on Computer Vision",
publisher = "IEEE, Institute of Electrical and Electronics Engineers",
pages = "2515--2523",
editor = "Ruzena Bajcsy and Greg Hager and Yi Ma",
booktitle = "2015 International Conference on Computer Vision, ICCV 2015",
address = "United States",

}

An, S, Hayat, M, Khan, S, Bennamoun, M, Boussaid, F & Sohel, F 2015, Contractive rectifier networks for nonlinear maximum margin classification. in R Bajcsy, G Hager & Y Ma (eds), 2015 International Conference on Computer Vision, ICCV 2015., 7410646, Proceedings of the IEEE International Conference on Computer Vision, vol. 2015 International Conference on Computer Vision, ICCV 2015, IEEE, Institute of Electrical and Electronics Engineers, Santiago, pp. 2515-2523, 2015 IEEE International Conference on Computer Vision, Santiago, Chile, 11/12/15. https://doi.org/10.1109/iccv.2015.289

Contractive rectifier networks for nonlinear maximum margin classification. / An, Senjian; Hayat, Munawar; Khan, Salam; Bennamoun, Mohammed; Boussaid, Farid; Sohel, Ferdous.

2015 International Conference on Computer Vision, ICCV 2015. ed. / Ruzena Bajcsy; Greg Hager; Yi Ma. Santiago : IEEE, Institute of Electrical and Electronics Engineers, 2015. p. 2515-2523 7410646 (Proceedings of the IEEE International Conference on Computer Vision; Vol. 2015 International Conference on Computer Vision, ICCV 2015).

Research output: A Conference proceeding or a Chapter in BookConference contribution

TY - GEN

T1 - Contractive rectifier networks for nonlinear maximum margin classification

AU - An, Senjian

AU - Hayat, Munawar

AU - Khan, Salam

AU - Bennamoun, Mohammed

AU - Boussaid, Farid

AU - Sohel, Ferdous

PY - 2015

Y1 - 2015

N2 - To find the optimal nonlinear separating boundary with maximum margin in the input data space, this paper proposes Contractive Rectifier Networks (CRNs), wherein the hidden-layer transformations are restricted to be contraction mappings. The contractive constraints ensure that the achieved separating margin in the input space is larger than or equal to the separating margin in the output layer. The training of the proposed CRNs is formulated as a linear support vector machine (SVM) in the output layer, combined with two or more contractive hidden layers. Effective algorithms have been proposed to address the optimization challenges arising from contraction constraints. Experimental results on MNIST, CIFAR-10, CIFAR-100 and MIT-67 datasets demonstrate that the proposed contractive rectifier networks consistently outperform their conventional unconstrained rectifier network counterparts

AB - To find the optimal nonlinear separating boundary with maximum margin in the input data space, this paper proposes Contractive Rectifier Networks (CRNs), wherein the hidden-layer transformations are restricted to be contraction mappings. The contractive constraints ensure that the achieved separating margin in the input space is larger than or equal to the separating margin in the output layer. The training of the proposed CRNs is formulated as a linear support vector machine (SVM) in the output layer, combined with two or more contractive hidden layers. Effective algorithms have been proposed to address the optimization challenges arising from contraction constraints. Experimental results on MNIST, CIFAR-10, CIFAR-100 and MIT-67 datasets demonstrate that the proposed contractive rectifier networks consistently outperform their conventional unconstrained rectifier network counterparts

KW - Deep learning

KW - Pattern recognition

KW - Computer vision

UR - http://www.scopus.com/inward/record.url?scp=84973923023&partnerID=8YFLogxK

UR - http://www.mendeley.com/research/contractive-rectifier-networks-nonlinear-maximum-margin-classification

U2 - 10.1109/iccv.2015.289

DO - 10.1109/iccv.2015.289

M3 - Conference contribution

SN - 9781467383912

T3 - Proceedings of the IEEE International Conference on Computer Vision

SP - 2515

EP - 2523

BT - 2015 International Conference on Computer Vision, ICCV 2015

A2 - Bajcsy, Ruzena

A2 - Hager, Greg

A2 - Ma, Yi

PB - IEEE, Institute of Electrical and Electronics Engineers

CY - Santiago

ER -

An S, Hayat M, Khan S, Bennamoun M, Boussaid F, Sohel F. Contractive rectifier networks for nonlinear maximum margin classification. In Bajcsy R, Hager G, Ma Y, editors, 2015 International Conference on Computer Vision, ICCV 2015. Santiago: IEEE, Institute of Electrical and Electronics Engineers. 2015. p. 2515-2523. 7410646. (Proceedings of the IEEE International Conference on Computer Vision). https://doi.org/10.1109/iccv.2015.289