Stereo 3D Lip Tracking

Gareth Loy, Roland GOECKE, Sebastian Rougeaux, Alexander Zelinsky

    Research output: A Conference proceeding or a Chapter in BookConference contribution

    Abstract

    A system is presented that tracks in 3D a person's unadorned lips, and outputs the 3D locations of the mouth corners and ten points describing the outer lip contour. This output is suitable for audio visual speech processing, 3D animation, or expression recognition. A stereo head tracker is used to track the subject's head, allowing for robust performance whilst the subject's head is moving and turning with respect to the cameras. The head pose is used in conjunction with the novel adaptable templates to generate a robust estimate of the deforming mouth corner locations. A 3D geometric model is used to generate search paths for key points on the outer lip contour which are subsequently located using adaptable templates and geometric constraints. The system is demonstrated robustly tracking the head pose and 3D mouth shape on a person speaking while moving his head.
    Original languageEnglish
    Title of host publicationProceedings of the 6th International Conference on Control, Automation, Robotics and Vision ICARCV2000
    Subtitle of host publicationIntelligent Automation in the New Millennium
    Place of PublicationSingapore
    PublisherNanyang Technological University
    PagesCD-ROM
    Number of pages6
    ISBN (Electronic)9810434456
    Publication statusPublished - 5 Dec 2000
    EventInternational Conference on Control, Automation, Robotics and Vision -
    Duration: 1 Jan 2011 → …

    Conference

    ConferenceInternational Conference on Control, Automation, Robotics and Vision
    Abbreviated titleICARCV
    Period1/01/11 → …

    Fingerprint

    Speech processing
    Animation
    Cameras

    Cite this

    Loy, G., GOECKE, R., Rougeaux, S., & Zelinsky, A. (2000). Stereo 3D Lip Tracking. In Proceedings of the 6th International Conference on Control, Automation, Robotics and Vision ICARCV2000: Intelligent Automation in the New Millennium (pp. CD-ROM). Singapore: Nanyang Technological University.
    Loy, Gareth ; GOECKE, Roland ; Rougeaux, Sebastian ; Zelinsky, Alexander. / Stereo 3D Lip Tracking. Proceedings of the 6th International Conference on Control, Automation, Robotics and Vision ICARCV2000: Intelligent Automation in the New Millennium. Singapore : Nanyang Technological University, 2000. pp. CD-ROM
    @inproceedings{49c5ecd7ff724c4f9b3a6dd9072afe65,
    title = "Stereo 3D Lip Tracking",
    abstract = "A system is presented that tracks in 3D a person's unadorned lips, and outputs the 3D locations of the mouth corners and ten points describing the outer lip contour. This output is suitable for audio visual speech processing, 3D animation, or expression recognition. A stereo head tracker is used to track the subject's head, allowing for robust performance whilst the subject's head is moving and turning with respect to the cameras. The head pose is used in conjunction with the novel adaptable templates to generate a robust estimate of the deforming mouth corner locations. A 3D geometric model is used to generate search paths for key points on the outer lip contour which are subsequently located using adaptable templates and geometric constraints. The system is demonstrated robustly tracking the head pose and 3D mouth shape on a person speaking while moving his head.",
    keywords = "Lip tracking, stereo image processing",
    author = "Gareth Loy and Roland GOECKE and Sebastian Rougeaux and Alexander Zelinsky",
    year = "2000",
    month = "12",
    day = "5",
    language = "English",
    pages = "CD--ROM",
    booktitle = "Proceedings of the 6th International Conference on Control, Automation, Robotics and Vision ICARCV2000",
    publisher = "Nanyang Technological University",

    }

    Loy, G, GOECKE, R, Rougeaux, S & Zelinsky, A 2000, Stereo 3D Lip Tracking. in Proceedings of the 6th International Conference on Control, Automation, Robotics and Vision ICARCV2000: Intelligent Automation in the New Millennium. Nanyang Technological University, Singapore, pp. CD-ROM, International Conference on Control, Automation, Robotics and Vision, 1/01/11.

    Stereo 3D Lip Tracking. / Loy, Gareth; GOECKE, Roland; Rougeaux, Sebastian; Zelinsky, Alexander.

    Proceedings of the 6th International Conference on Control, Automation, Robotics and Vision ICARCV2000: Intelligent Automation in the New Millennium. Singapore : Nanyang Technological University, 2000. p. CD-ROM.

    Research output: A Conference proceeding or a Chapter in BookConference contribution

    TY - GEN

    T1 - Stereo 3D Lip Tracking

    AU - Loy, Gareth

    AU - GOECKE, Roland

    AU - Rougeaux, Sebastian

    AU - Zelinsky, Alexander

    PY - 2000/12/5

    Y1 - 2000/12/5

    N2 - A system is presented that tracks in 3D a person's unadorned lips, and outputs the 3D locations of the mouth corners and ten points describing the outer lip contour. This output is suitable for audio visual speech processing, 3D animation, or expression recognition. A stereo head tracker is used to track the subject's head, allowing for robust performance whilst the subject's head is moving and turning with respect to the cameras. The head pose is used in conjunction with the novel adaptable templates to generate a robust estimate of the deforming mouth corner locations. A 3D geometric model is used to generate search paths for key points on the outer lip contour which are subsequently located using adaptable templates and geometric constraints. The system is demonstrated robustly tracking the head pose and 3D mouth shape on a person speaking while moving his head.

    AB - A system is presented that tracks in 3D a person's unadorned lips, and outputs the 3D locations of the mouth corners and ten points describing the outer lip contour. This output is suitable for audio visual speech processing, 3D animation, or expression recognition. A stereo head tracker is used to track the subject's head, allowing for robust performance whilst the subject's head is moving and turning with respect to the cameras. The head pose is used in conjunction with the novel adaptable templates to generate a robust estimate of the deforming mouth corner locations. A 3D geometric model is used to generate search paths for key points on the outer lip contour which are subsequently located using adaptable templates and geometric constraints. The system is demonstrated robustly tracking the head pose and 3D mouth shape on a person speaking while moving his head.

    KW - Lip tracking

    KW - stereo image processing

    M3 - Conference contribution

    SP - CD-ROM

    BT - Proceedings of the 6th International Conference on Control, Automation, Robotics and Vision ICARCV2000

    PB - Nanyang Technological University

    CY - Singapore

    ER -

    Loy G, GOECKE R, Rougeaux S, Zelinsky A. Stereo 3D Lip Tracking. In Proceedings of the 6th International Conference on Control, Automation, Robotics and Vision ICARCV2000: Intelligent Automation in the New Millennium. Singapore: Nanyang Technological University. 2000. p. CD-ROM