Text to visual synthesis with appearance models

Javier Melenchón, Fernando De La Torre, Igfnasi Iriondo, Francesc Alías, Elisa Martinez, Luis Vicent

Research output: A Conference proceeding or a Chapter in BookConference contribution

5 Citations (Scopus)

Abstract

This paper presents a new method named text to visual synthesis with appearance models (TEVISAM) for generating videorealistic talking heads. In a first step, the system learns a person-specific facial appearance model (PSF AM) automatically. PSF AM allows modeling all facial components (e.g. eyes, mouth, etc) independently and it will be used to animate die face from the input text dynamically. As reported by other researches, one of the key aspects in visual synthesis is the coarticulation effect. To solve such a problem, we introduce a new interpolation method in the high dimensional space of appearance allowing to create photorealistic and videorealistic avatars. In this work, preliminary experiments synthesizing virtual avatars from text are reported. Summarizing, in this paper we introduce three novelties: first, we make use of color PSFAM to animate virtual avatars; second, we introduce a non-linear high dimensional interpolation to achieve videorealistic animations; finally, this method allows to generate new expressions modeling the different facial elements.

Original languageEnglish
Title of host publicationIEEE International Conference on Image Processing
PublisherIEEE
Pages237-240
Number of pages4
Volume1
ISBN (Print)9780780377508
DOIs
Publication statusPublished - 14 Sep 2003
Externally publishedYes
EventProceedings: 2003 International Conference on Image Processing, ICIP-2003 - Barcelona, Spain
Duration: 14 Sep 200317 Sep 2003

Conference

ConferenceProceedings: 2003 International Conference on Image Processing, ICIP-2003
CountrySpain
CityBarcelona
Period14/09/0317/09/03

Fingerprint

Interpolation
Animation
Color
Experiments

Cite this

Melenchón, J., La Torre, F. D., Iriondo, I., Alías, F., Martinez, E., & Vicent, L. (2003). Text to visual synthesis with appearance models. In IEEE International Conference on Image Processing (Vol. 1, pp. 237-240). IEEE. https://doi.org/10.1109/ICIP.2003.1246942
Melenchón, Javier ; La Torre, Fernando De ; Iriondo, Igfnasi ; Alías, Francesc ; Martinez, Elisa ; Vicent, Luis. / Text to visual synthesis with appearance models. IEEE International Conference on Image Processing. Vol. 1 IEEE, 2003. pp. 237-240
@inproceedings{4101ebebaf7b4890a8db90652227848f,
title = "Text to visual synthesis with appearance models",
abstract = "This paper presents a new method named text to visual synthesis with appearance models (TEVISAM) for generating videorealistic talking heads. In a first step, the system learns a person-specific facial appearance model (PSF AM) automatically. PSF AM allows modeling all facial components (e.g. eyes, mouth, etc) independently and it will be used to animate die face from the input text dynamically. As reported by other researches, one of the key aspects in visual synthesis is the coarticulation effect. To solve such a problem, we introduce a new interpolation method in the high dimensional space of appearance allowing to create photorealistic and videorealistic avatars. In this work, preliminary experiments synthesizing virtual avatars from text are reported. Summarizing, in this paper we introduce three novelties: first, we make use of color PSFAM to animate virtual avatars; second, we introduce a non-linear high dimensional interpolation to achieve videorealistic animations; finally, this method allows to generate new expressions modeling the different facial elements.",
keywords = "Appearance model, Visual synthesis",
author = "Javier Melench{\'o}n and {La Torre}, {Fernando De} and Igfnasi Iriondo and Francesc Al{\'i}as and Elisa Martinez and Luis Vicent",
year = "2003",
month = "9",
day = "14",
doi = "10.1109/ICIP.2003.1246942",
language = "English",
isbn = "9780780377508",
volume = "1",
pages = "237--240",
booktitle = "IEEE International Conference on Image Processing",
publisher = "IEEE",

}

Melenchón, J, La Torre, FD, Iriondo, I, Alías, F, Martinez, E & Vicent, L 2003, Text to visual synthesis with appearance models. in IEEE International Conference on Image Processing. vol. 1, IEEE, pp. 237-240, Proceedings: 2003 International Conference on Image Processing, ICIP-2003, Barcelona, Spain, 14/09/03. https://doi.org/10.1109/ICIP.2003.1246942

Text to visual synthesis with appearance models. / Melenchón, Javier; La Torre, Fernando De; Iriondo, Igfnasi; Alías, Francesc; Martinez, Elisa; Vicent, Luis.

IEEE International Conference on Image Processing. Vol. 1 IEEE, 2003. p. 237-240.

Research output: A Conference proceeding or a Chapter in BookConference contribution

TY - GEN

T1 - Text to visual synthesis with appearance models

AU - Melenchón, Javier

AU - La Torre, Fernando De

AU - Iriondo, Igfnasi

AU - Alías, Francesc

AU - Martinez, Elisa

AU - Vicent, Luis

PY - 2003/9/14

Y1 - 2003/9/14

N2 - This paper presents a new method named text to visual synthesis with appearance models (TEVISAM) for generating videorealistic talking heads. In a first step, the system learns a person-specific facial appearance model (PSF AM) automatically. PSF AM allows modeling all facial components (e.g. eyes, mouth, etc) independently and it will be used to animate die face from the input text dynamically. As reported by other researches, one of the key aspects in visual synthesis is the coarticulation effect. To solve such a problem, we introduce a new interpolation method in the high dimensional space of appearance allowing to create photorealistic and videorealistic avatars. In this work, preliminary experiments synthesizing virtual avatars from text are reported. Summarizing, in this paper we introduce three novelties: first, we make use of color PSFAM to animate virtual avatars; second, we introduce a non-linear high dimensional interpolation to achieve videorealistic animations; finally, this method allows to generate new expressions modeling the different facial elements.

AB - This paper presents a new method named text to visual synthesis with appearance models (TEVISAM) for generating videorealistic talking heads. In a first step, the system learns a person-specific facial appearance model (PSF AM) automatically. PSF AM allows modeling all facial components (e.g. eyes, mouth, etc) independently and it will be used to animate die face from the input text dynamically. As reported by other researches, one of the key aspects in visual synthesis is the coarticulation effect. To solve such a problem, we introduce a new interpolation method in the high dimensional space of appearance allowing to create photorealistic and videorealistic avatars. In this work, preliminary experiments synthesizing virtual avatars from text are reported. Summarizing, in this paper we introduce three novelties: first, we make use of color PSFAM to animate virtual avatars; second, we introduce a non-linear high dimensional interpolation to achieve videorealistic animations; finally, this method allows to generate new expressions modeling the different facial elements.

KW - Appearance model

KW - Visual synthesis

UR - http://www.scopus.com/inward/record.url?scp=0345097527&partnerID=8YFLogxK

U2 - 10.1109/ICIP.2003.1246942

DO - 10.1109/ICIP.2003.1246942

M3 - Conference contribution

SN - 9780780377508

VL - 1

SP - 237

EP - 240

BT - IEEE International Conference on Image Processing

PB - IEEE

ER -

Melenchón J, La Torre FD, Iriondo I, Alías F, Martinez E, Vicent L. Text to visual synthesis with appearance models. In IEEE International Conference on Image Processing. Vol. 1. IEEE. 2003. p. 237-240 https://doi.org/10.1109/ICIP.2003.1246942