Facial Performance Transfer via Deformable Models and Parametric Correspondence

Akshay Asthana, Miles De La Hunty, Abhinav Dhall, Roland Goecke

    Research output: Contribution to journalArticle

    12 Citations (Scopus)
    1 Downloads (Pure)

    Abstract

    The issue of transferring facial performance from one person's face to another's has been an area of interest for the movie industry and the computer graphics community for quite some time. In recent years, deformable face models, such as the Active Appearance Model (AAM), have made it possible to track and synthesize faces in real time. Not surprisingly, deformable face model-based approaches for facial performance transfer have gained tremendous interest in the computer vision and graphics community. In this paper, we focus on the problem of real-time facial performance transfer using the AAM framework. We propose a novel approach of learning the mapping between the parameters of two completely independent AAMs, using them to facilitate the facial performance transfer in a more realistic manner than previous approaches. The main advantage of modeling this parametric correspondence is that it allows a "meaningful” transfer of both the nonrigid shape and texture across faces irrespective of the speakers' gender, shape, and size of the faces, and illumination conditions. We explore linear and nonlinear methods for modeling the parametric correspondence between the AAMs and show that the sparse linear regression method performs the best. Moreover, we show the utility of the proposed framework for a cross-language facial performance transfer that is an area of interest for the movie dubbing industry.
    Original languageEnglish
    Pages (from-to)1511-1519
    Number of pages9
    JournalIEEE Transactions on Visualization and Computer Graphics
    Volume18
    Issue number9
    DOIs
    Publication statusPublished - Sep 2012

    Fingerprint

    Computer graphics
    Linear regression
    Computer vision
    Industry
    Textures
    Lighting

    Cite this

    Asthana, Akshay ; De La Hunty, Miles ; Dhall, Abhinav ; Goecke, Roland. / Facial Performance Transfer via Deformable Models and Parametric Correspondence. In: IEEE Transactions on Visualization and Computer Graphics. 2012 ; Vol. 18, No. 9. pp. 1511-1519.
    @article{317f3746370247888d12ca3ce5b6e3b9,
    title = "Facial Performance Transfer via Deformable Models and Parametric Correspondence",
    abstract = "The issue of transferring facial performance from one person's face to another's has been an area of interest for the movie industry and the computer graphics community for quite some time. In recent years, deformable face models, such as the Active Appearance Model (AAM), have made it possible to track and synthesize faces in real time. Not surprisingly, deformable face model-based approaches for facial performance transfer have gained tremendous interest in the computer vision and graphics community. In this paper, we focus on the problem of real-time facial performance transfer using the AAM framework. We propose a novel approach of learning the mapping between the parameters of two completely independent AAMs, using them to facilitate the facial performance transfer in a more realistic manner than previous approaches. The main advantage of modeling this parametric correspondence is that it allows a {"}meaningful” transfer of both the nonrigid shape and texture across faces irrespective of the speakers' gender, shape, and size of the faces, and illumination conditions. We explore linear and nonlinear methods for modeling the parametric correspondence between the AAMs and show that the sparse linear regression method performs the best. Moreover, we show the utility of the proposed framework for a cross-language facial performance transfer that is an area of interest for the movie dubbing industry.",
    keywords = "Facial expression transfer, Deformable models, Parametric Correspondence, active apperance models",
    author = "Akshay Asthana and {De La Hunty}, Miles and Abhinav Dhall and Roland Goecke",
    year = "2012",
    month = "9",
    doi = "10.1109/TVCG.2011.157",
    language = "English",
    volume = "18",
    pages = "1511--1519",
    journal = "IEEE Transactions on Visualization and Computer Graphics",
    issn = "1077-2626",
    publisher = "IEEE, Institute of Electrical and Electronics Engineers",
    number = "9",

    }

    Facial Performance Transfer via Deformable Models and Parametric Correspondence. / Asthana, Akshay; De La Hunty, Miles; Dhall, Abhinav; Goecke, Roland.

    In: IEEE Transactions on Visualization and Computer Graphics, Vol. 18, No. 9, 09.2012, p. 1511-1519.

    Research output: Contribution to journalArticle

    TY - JOUR

    T1 - Facial Performance Transfer via Deformable Models and Parametric Correspondence

    AU - Asthana, Akshay

    AU - De La Hunty, Miles

    AU - Dhall, Abhinav

    AU - Goecke, Roland

    PY - 2012/9

    Y1 - 2012/9

    N2 - The issue of transferring facial performance from one person's face to another's has been an area of interest for the movie industry and the computer graphics community for quite some time. In recent years, deformable face models, such as the Active Appearance Model (AAM), have made it possible to track and synthesize faces in real time. Not surprisingly, deformable face model-based approaches for facial performance transfer have gained tremendous interest in the computer vision and graphics community. In this paper, we focus on the problem of real-time facial performance transfer using the AAM framework. We propose a novel approach of learning the mapping between the parameters of two completely independent AAMs, using them to facilitate the facial performance transfer in a more realistic manner than previous approaches. The main advantage of modeling this parametric correspondence is that it allows a "meaningful” transfer of both the nonrigid shape and texture across faces irrespective of the speakers' gender, shape, and size of the faces, and illumination conditions. We explore linear and nonlinear methods for modeling the parametric correspondence between the AAMs and show that the sparse linear regression method performs the best. Moreover, we show the utility of the proposed framework for a cross-language facial performance transfer that is an area of interest for the movie dubbing industry.

    AB - The issue of transferring facial performance from one person's face to another's has been an area of interest for the movie industry and the computer graphics community for quite some time. In recent years, deformable face models, such as the Active Appearance Model (AAM), have made it possible to track and synthesize faces in real time. Not surprisingly, deformable face model-based approaches for facial performance transfer have gained tremendous interest in the computer vision and graphics community. In this paper, we focus on the problem of real-time facial performance transfer using the AAM framework. We propose a novel approach of learning the mapping between the parameters of two completely independent AAMs, using them to facilitate the facial performance transfer in a more realistic manner than previous approaches. The main advantage of modeling this parametric correspondence is that it allows a "meaningful” transfer of both the nonrigid shape and texture across faces irrespective of the speakers' gender, shape, and size of the faces, and illumination conditions. We explore linear and nonlinear methods for modeling the parametric correspondence between the AAMs and show that the sparse linear regression method performs the best. Moreover, we show the utility of the proposed framework for a cross-language facial performance transfer that is an area of interest for the movie dubbing industry.

    KW - Facial expression transfer

    KW - Deformable models

    KW - Parametric Correspondence

    KW - active apperance models

    U2 - 10.1109/TVCG.2011.157

    DO - 10.1109/TVCG.2011.157

    M3 - Article

    VL - 18

    SP - 1511

    EP - 1519

    JO - IEEE Transactions on Visualization and Computer Graphics

    JF - IEEE Transactions on Visualization and Computer Graphics

    SN - 1077-2626

    IS - 9

    ER -