Adversarial Training of Variational Auto-Encoders for High Fidelity Image Generation

Salman H. Khan, Munawar Hayat, Nick Barnes

Research output: A Conference proceeding or a Chapter in BookConference contribution

3 Citations (Scopus)

Abstract

Variational auto-encoders (VAEs) provide an attractive solution to image generation problem. However, they tend to produce blurred and over-smoothed images due to their dependence on pixel-wise reconstruction loss. This paper introduces a new approach to alleviate this problem in the VAE based generative models. Our model simultaneously learns to match the data, reconstruction loss and the latent distributions of real and fake images to improve the quality of generated samples. To compute the loss distributions, we introduce an auto-encoder based discriminator model which allows an adversarial learning procedure. The discriminator in our model also provides perceptual guidance to the VAE by matching the learned similarity metric of the real and fake samples in the latent space. To stabilize the overall training process, our model uses an error feedback approach to maintain the equilibrium between competing networks in the model. Our experiments show that the generated samples from our proposed model exhibit a diverse set of attributes and facial expressions and scale up to highresolution images very well.

Original languageEnglish
Title of host publicationProceedings - 2018 IEEE Winter Conference on Applications of Computer Vision, WACV 2018
PublisherIEEE, Institute of Electrical and Electronics Engineers
Pages1312-1320
Number of pages9
Volume2018-January
ISBN (Electronic)9781538648865
ISBN (Print)9781538648872
DOIs
Publication statusPublished - 12 Mar 2018
Event18th IEEE Winter Conference on Applications of Computer Vision, WACV 2018 - Lake Tahoe, United States
Duration: 12 Mar 201815 Mar 2018

Conference

Conference18th IEEE Winter Conference on Applications of Computer Vision, WACV 2018
CountryUnited States
CityLake Tahoe
Period12/03/1815/03/18

Fingerprint

Discriminators
Pixels
Feedback
Experiments

Cite this

Khan, S. H., Hayat, M., & Barnes, N. (2018). Adversarial Training of Variational Auto-Encoders for High Fidelity Image Generation. In Proceedings - 2018 IEEE Winter Conference on Applications of Computer Vision, WACV 2018 (Vol. 2018-January, pp. 1312-1320). IEEE, Institute of Electrical and Electronics Engineers. https://doi.org/10.1109/WACV.2018.00148
Khan, Salman H. ; Hayat, Munawar ; Barnes, Nick. / Adversarial Training of Variational Auto-Encoders for High Fidelity Image Generation. Proceedings - 2018 IEEE Winter Conference on Applications of Computer Vision, WACV 2018. Vol. 2018-January IEEE, Institute of Electrical and Electronics Engineers, 2018. pp. 1312-1320
@inproceedings{5080cdcc39c145eba42870a61074dd65,
title = "Adversarial Training of Variational Auto-Encoders for High Fidelity Image Generation",
abstract = "Variational auto-encoders (VAEs) provide an attractive solution to image generation problem. However, they tend to produce blurred and over-smoothed images due to their dependence on pixel-wise reconstruction loss. This paper introduces a new approach to alleviate this problem in the VAE based generative models. Our model simultaneously learns to match the data, reconstruction loss and the latent distributions of real and fake images to improve the quality of generated samples. To compute the loss distributions, we introduce an auto-encoder based discriminator model which allows an adversarial learning procedure. The discriminator in our model also provides perceptual guidance to the VAE by matching the learned similarity metric of the real and fake samples in the latent space. To stabilize the overall training process, our model uses an error feedback approach to maintain the equilibrium between competing networks in the model. Our experiments show that the generated samples from our proposed model exhibit a diverse set of attributes and facial expressions and scale up to highresolution images very well.",
keywords = "Variational auto-encoders, Generative model, image generation, Adversarial training",
author = "Khan, {Salman H.} and Munawar Hayat and Nick Barnes",
year = "2018",
month = "3",
day = "12",
doi = "10.1109/WACV.2018.00148",
language = "English",
isbn = "9781538648872",
volume = "2018-January",
pages = "1312--1320",
booktitle = "Proceedings - 2018 IEEE Winter Conference on Applications of Computer Vision, WACV 2018",
publisher = "IEEE, Institute of Electrical and Electronics Engineers",
address = "United States",

}

Khan, SH, Hayat, M & Barnes, N 2018, Adversarial Training of Variational Auto-Encoders for High Fidelity Image Generation. in Proceedings - 2018 IEEE Winter Conference on Applications of Computer Vision, WACV 2018. vol. 2018-January, IEEE, Institute of Electrical and Electronics Engineers, pp. 1312-1320, 18th IEEE Winter Conference on Applications of Computer Vision, WACV 2018, Lake Tahoe, United States, 12/03/18. https://doi.org/10.1109/WACV.2018.00148

Adversarial Training of Variational Auto-Encoders for High Fidelity Image Generation. / Khan, Salman H.; Hayat, Munawar; Barnes, Nick.

Proceedings - 2018 IEEE Winter Conference on Applications of Computer Vision, WACV 2018. Vol. 2018-January IEEE, Institute of Electrical and Electronics Engineers, 2018. p. 1312-1320.

Research output: A Conference proceeding or a Chapter in BookConference contribution

TY - GEN

T1 - Adversarial Training of Variational Auto-Encoders for High Fidelity Image Generation

AU - Khan, Salman H.

AU - Hayat, Munawar

AU - Barnes, Nick

PY - 2018/3/12

Y1 - 2018/3/12

N2 - Variational auto-encoders (VAEs) provide an attractive solution to image generation problem. However, they tend to produce blurred and over-smoothed images due to their dependence on pixel-wise reconstruction loss. This paper introduces a new approach to alleviate this problem in the VAE based generative models. Our model simultaneously learns to match the data, reconstruction loss and the latent distributions of real and fake images to improve the quality of generated samples. To compute the loss distributions, we introduce an auto-encoder based discriminator model which allows an adversarial learning procedure. The discriminator in our model also provides perceptual guidance to the VAE by matching the learned similarity metric of the real and fake samples in the latent space. To stabilize the overall training process, our model uses an error feedback approach to maintain the equilibrium between competing networks in the model. Our experiments show that the generated samples from our proposed model exhibit a diverse set of attributes and facial expressions and scale up to highresolution images very well.

AB - Variational auto-encoders (VAEs) provide an attractive solution to image generation problem. However, they tend to produce blurred and over-smoothed images due to their dependence on pixel-wise reconstruction loss. This paper introduces a new approach to alleviate this problem in the VAE based generative models. Our model simultaneously learns to match the data, reconstruction loss and the latent distributions of real and fake images to improve the quality of generated samples. To compute the loss distributions, we introduce an auto-encoder based discriminator model which allows an adversarial learning procedure. The discriminator in our model also provides perceptual guidance to the VAE by matching the learned similarity metric of the real and fake samples in the latent space. To stabilize the overall training process, our model uses an error feedback approach to maintain the equilibrium between competing networks in the model. Our experiments show that the generated samples from our proposed model exhibit a diverse set of attributes and facial expressions and scale up to highresolution images very well.

KW - Variational auto-encoders

KW - Generative model

KW - image generation

KW - Adversarial training

UR - http://www.scopus.com/inward/record.url?scp=85050949653&partnerID=8YFLogxK

U2 - 10.1109/WACV.2018.00148

DO - 10.1109/WACV.2018.00148

M3 - Conference contribution

SN - 9781538648872

VL - 2018-January

SP - 1312

EP - 1320

BT - Proceedings - 2018 IEEE Winter Conference on Applications of Computer Vision, WACV 2018

PB - IEEE, Institute of Electrical and Electronics Engineers

ER -

Khan SH, Hayat M, Barnes N. Adversarial Training of Variational Auto-Encoders for High Fidelity Image Generation. In Proceedings - 2018 IEEE Winter Conference on Applications of Computer Vision, WACV 2018. Vol. 2018-January. IEEE, Institute of Electrical and Electronics Engineers. 2018. p. 1312-1320 https://doi.org/10.1109/WACV.2018.00148