Play with me - Measuring a child's engagement in a social interaction

Shyam RAJAGOPALAN, Ramana ORUGANTI, Roland GOECKE, Agata Rozga

Research output: A Conference proceeding or a Chapter in BookConference contribution

7 Citations (Scopus)

Abstract

Due to the challenges in automatically observing child behaviour in a social interaction, an automatic extraction of high-level features, such as head poses and hand gestures, is difficult and noisy, leading to an inaccurate model. Hence, the feasibility of using easily obtainable low-level optical flow based features is investigated in this work. A comparative study involving high-level features, baseline annotations of multiple modalities and the low-level features is carried out. Optical flow based hidden structure learning of behaviours is strongly discriminatory in predicting a child’s engagement level in a social interaction. A two-stage approach of discovering the hidden structures using Hidden Conditional Random Fields, followed by learning an SVM-based model on the hidden state marginals is proposed. This is validated by conducting experiments on the Multimodal Dyadic Behaviour Dataset and the results indicate a state of the art classification performance. The insights drawn from this study indicate the robustness of the low-level feature approach towards engagement behaviour
modelling and can be a good substitute in the absence of accurate high-level features.
Original languageEnglish
Title of host publication2015 11th IEEE International conference and workshop on Automatic face and gesture recognition (FG 2015)
EditorsKevin Bowyer, Ales Leonardo, Jeff Cohn
Place of PublicationLjubljana, Slovenia
PublisherIEEE
Pages1-8
Number of pages8
Volume1
ISBN (Electronic)9781479960262
DOIs
Publication statusPublished - 4 May 2015
Event11th IEEE International conference and workshop on Automatic face and gesture recognition 2015 - Ljubljana, Ljubljana, Slovenia
Duration: 4 May 20158 May 2015
http://www.fg2015.org/ (Conference website)

Publication series

Name2015 11th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition, FG 2015

Conference

Conference11th IEEE International conference and workshop on Automatic face and gesture recognition 2015
CountrySlovenia
CityLjubljana
Period4/05/158/05/15
OtherThe IEEE conference series on Automatic Face and Gesture Recognition is the premier international forum for research in image and video-based face, gesture, and body movement recognition. Its broad scope includes: advances in fundamental computer vision, pattern recognition and computer graphics; machine learning techniques relevant to face, gesture, and body motion; new algorithms and applications. The conference presents research that advances the state-of-the-art in these and related areas, leading to new capabilities in various application domains
Internet address

Fingerprint

Optical flows
Experiments

Cite this

RAJAGOPALAN, S., ORUGANTI, R., GOECKE, R., & Rozga, A. (2015). Play with me - Measuring a child's engagement in a social interaction. In K. Bowyer, A. Leonardo, & J. Cohn (Eds.), 2015 11th IEEE International conference and workshop on Automatic face and gesture recognition (FG 2015) (Vol. 1, pp. 1-8). (2015 11th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition, FG 2015). Ljubljana, Slovenia: IEEE. https://doi.org/10.1109/fg.2015.7163129
RAJAGOPALAN, Shyam ; ORUGANTI, Ramana ; GOECKE, Roland ; Rozga, Agata. / Play with me - Measuring a child's engagement in a social interaction. 2015 11th IEEE International conference and workshop on Automatic face and gesture recognition (FG 2015). editor / Kevin Bowyer ; Ales Leonardo ; Jeff Cohn. Vol. 1 Ljubljana, Slovenia : IEEE, 2015. pp. 1-8 (2015 11th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition, FG 2015).
@inproceedings{861119d5f75c4faaa67230911c54d35a,
title = "Play with me - Measuring a child's engagement in a social interaction",
abstract = "Due to the challenges in automatically observing child behaviour in a social interaction, an automatic extraction of high-level features, such as head poses and hand gestures, is difficult and noisy, leading to an inaccurate model. Hence, the feasibility of using easily obtainable low-level optical flow based features is investigated in this work. A comparative study involving high-level features, baseline annotations of multiple modalities and the low-level features is carried out. Optical flow based hidden structure learning of behaviours is strongly discriminatory in predicting a child’s engagement level in a social interaction. A two-stage approach of discovering the hidden structures using Hidden Conditional Random Fields, followed by learning an SVM-based model on the hidden state marginals is proposed. This is validated by conducting experiments on the Multimodal Dyadic Behaviour Dataset and the results indicate a state of the art classification performance. The insights drawn from this study indicate the robustness of the low-level feature approach towards engagement behaviourmodelling and can be a good substitute in the absence of accurate high-level features.",
keywords = "Social interactions, computer-modeling, hidden-markov-models",
author = "Shyam RAJAGOPALAN and Ramana ORUGANTI and Roland GOECKE and Agata Rozga",
year = "2015",
month = "5",
day = "4",
doi = "10.1109/fg.2015.7163129",
language = "English",
volume = "1",
series = "2015 11th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition, FG 2015",
publisher = "IEEE",
pages = "1--8",
editor = "Kevin Bowyer and Ales Leonardo and Jeff Cohn",
booktitle = "2015 11th IEEE International conference and workshop on Automatic face and gesture recognition (FG 2015)",

}

RAJAGOPALAN, S, ORUGANTI, R, GOECKE, R & Rozga, A 2015, Play with me - Measuring a child's engagement in a social interaction. in K Bowyer, A Leonardo & J Cohn (eds), 2015 11th IEEE International conference and workshop on Automatic face and gesture recognition (FG 2015). vol. 1, 2015 11th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition, FG 2015, IEEE, Ljubljana, Slovenia, pp. 1-8, 11th IEEE International conference and workshop on Automatic face and gesture recognition 2015, Ljubljana, Slovenia, 4/05/15. https://doi.org/10.1109/fg.2015.7163129

Play with me - Measuring a child's engagement in a social interaction. / RAJAGOPALAN, Shyam; ORUGANTI, Ramana; GOECKE, Roland; Rozga, Agata.

2015 11th IEEE International conference and workshop on Automatic face and gesture recognition (FG 2015). ed. / Kevin Bowyer; Ales Leonardo; Jeff Cohn. Vol. 1 Ljubljana, Slovenia : IEEE, 2015. p. 1-8 (2015 11th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition, FG 2015).

Research output: A Conference proceeding or a Chapter in BookConference contribution

TY - GEN

T1 - Play with me - Measuring a child's engagement in a social interaction

AU - RAJAGOPALAN, Shyam

AU - ORUGANTI, Ramana

AU - GOECKE, Roland

AU - Rozga, Agata

PY - 2015/5/4

Y1 - 2015/5/4

N2 - Due to the challenges in automatically observing child behaviour in a social interaction, an automatic extraction of high-level features, such as head poses and hand gestures, is difficult and noisy, leading to an inaccurate model. Hence, the feasibility of using easily obtainable low-level optical flow based features is investigated in this work. A comparative study involving high-level features, baseline annotations of multiple modalities and the low-level features is carried out. Optical flow based hidden structure learning of behaviours is strongly discriminatory in predicting a child’s engagement level in a social interaction. A two-stage approach of discovering the hidden structures using Hidden Conditional Random Fields, followed by learning an SVM-based model on the hidden state marginals is proposed. This is validated by conducting experiments on the Multimodal Dyadic Behaviour Dataset and the results indicate a state of the art classification performance. The insights drawn from this study indicate the robustness of the low-level feature approach towards engagement behaviourmodelling and can be a good substitute in the absence of accurate high-level features.

AB - Due to the challenges in automatically observing child behaviour in a social interaction, an automatic extraction of high-level features, such as head poses and hand gestures, is difficult and noisy, leading to an inaccurate model. Hence, the feasibility of using easily obtainable low-level optical flow based features is investigated in this work. A comparative study involving high-level features, baseline annotations of multiple modalities and the low-level features is carried out. Optical flow based hidden structure learning of behaviours is strongly discriminatory in predicting a child’s engagement level in a social interaction. A two-stage approach of discovering the hidden structures using Hidden Conditional Random Fields, followed by learning an SVM-based model on the hidden state marginals is proposed. This is validated by conducting experiments on the Multimodal Dyadic Behaviour Dataset and the results indicate a state of the art classification performance. The insights drawn from this study indicate the robustness of the low-level feature approach towards engagement behaviourmodelling and can be a good substitute in the absence of accurate high-level features.

KW - Social interactions

KW - computer-modeling

KW - hidden-markov-models

UR - http://www.scopus.com/inward/record.url?scp=84944930963&partnerID=8YFLogxK

U2 - 10.1109/fg.2015.7163129

DO - 10.1109/fg.2015.7163129

M3 - Conference contribution

VL - 1

T3 - 2015 11th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition, FG 2015

SP - 1

EP - 8

BT - 2015 11th IEEE International conference and workshop on Automatic face and gesture recognition (FG 2015)

A2 - Bowyer, Kevin

A2 - Leonardo, Ales

A2 - Cohn, Jeff

PB - IEEE

CY - Ljubljana, Slovenia

ER -

RAJAGOPALAN S, ORUGANTI R, GOECKE R, Rozga A. Play with me - Measuring a child's engagement in a social interaction. In Bowyer K, Leonardo A, Cohn J, editors, 2015 11th IEEE International conference and workshop on Automatic face and gesture recognition (FG 2015). Vol. 1. Ljubljana, Slovenia: IEEE. 2015. p. 1-8. (2015 11th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition, FG 2015). https://doi.org/10.1109/fg.2015.7163129