Play with me - Measuring a child's engagement in a social interaction

Shyam RAJAGOPALAN; Ramana ORUGANTI; Roland GOECKE; Agata Rozga

doi:10.1109/fg.2015.7163129

Play with me - Measuring a child's engagement in a social interaction

Shyam RAJAGOPALAN, Ramana ORUGANTI, Roland GOECKE, Agata Rozga

Information Systems

Research output: A Conference proceeding or a Chapter in Book › Conference contribution › peer-review

14 Citations (Scopus)

Abstract

Due to the challenges in automatically observing child behaviour in a social interaction, an automatic extraction of high-level features, such as head poses and hand gestures, is difficult and noisy, leading to an inaccurate model. Hence, the feasibility of using easily obtainable low-level optical flow based features is investigated in this work. A comparative study involving high-level features, baseline annotations of multiple modalities and the low-level features is carried out. Optical flow based hidden structure learning of behaviours is strongly discriminatory in predicting a child’s engagement level in a social interaction. A two-stage approach of discovering the hidden structures using Hidden Conditional Random Fields, followed by learning an SVM-based model on the hidden state marginals is proposed. This is validated by conducting experiments on the Multimodal Dyadic Behaviour Dataset and the results indicate a state of the art classification performance. The insights drawn from this study indicate the robustness of the low-level feature approach towards engagement behaviour
modelling and can be a good substitute in the absence of accurate high-level features.

Original language	English
Title of host publication	2015 11th IEEE International conference and workshop on Automatic face and gesture recognition (FG 2015)
Editors	Kevin Bowyer, Ales Leonardo, Jeff Cohn
Place of Publication	Ljubljana, Slovenia
Publisher	IEEE, Institute of Electrical and Electronics Engineers
Pages	1-8
Number of pages	8
Volume	1
ISBN (Electronic)	9781479960262
DOIs	https://doi.org/10.1109/fg.2015.7163129
Publication status	Published - 4 May 2015
Event	11th IEEE International conference and workshop on Automatic face and gesture recognition 2015 - Ljubljana, Ljubljana, Slovenia Duration: 4 May 2015 → 8 May 2015 http://www.fg2015.org/ (Conference website)

Publication series

Name	2015 11th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition, FG 2015

Conference

Conference	11th IEEE International conference and workshop on Automatic face and gesture recognition 2015
Country/Territory	Slovenia
City	Ljubljana
Period	4/05/15 → 8/05/15
Other	The IEEE conference series on Automatic Face and Gesture Recognition is the premier international forum for research in image and video-based face, gesture, and body movement recognition. Its broad scope includes: advances in fundamental computer vision, pattern recognition and computer graphics; machine learning techniques relevant to face, gesture, and body motion; new algorithms and applications. The conference presents research that advances the state-of-the-art in these and related areas, leading to new capabilities in various application domains
Internet address	http://www.fg2015.org/ (Conference website)

Access to Document

10.1109/fg.2015.7163129

Cite this

RAJAGOPALAN, S., ORUGANTI, R., GOECKE, R., & Rozga, A. (2015). Play with me - Measuring a child's engagement in a social interaction. In K. Bowyer, A. Leonardo, & J. Cohn (Eds.), 2015 11th IEEE International conference and workshop on Automatic face and gesture recognition (FG 2015) (Vol. 1, pp. 1-8). (2015 11th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition, FG 2015). IEEE, Institute of Electrical and Electronics Engineers. https://doi.org/10.1109/fg.2015.7163129

RAJAGOPALAN, Shyam ; ORUGANTI, Ramana ; GOECKE, Roland et al. / Play with me - Measuring a child's engagement in a social interaction. 2015 11th IEEE International conference and workshop on Automatic face and gesture recognition (FG 2015). editor / Kevin Bowyer ; Ales Leonardo ; Jeff Cohn. Vol. 1 Ljubljana, Slovenia : IEEE, Institute of Electrical and Electronics Engineers, 2015. pp. 1-8 (2015 11th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition, FG 2015).

@inproceedings{861119d5f75c4faaa67230911c54d35a,

title = "Play with me - Measuring a child's engagement in a social interaction",

abstract = "Due to the challenges in automatically observing child behaviour in a social interaction, an automatic extraction of high-level features, such as head poses and hand gestures, is difficult and noisy, leading to an inaccurate model. Hence, the feasibility of using easily obtainable low-level optical flow based features is investigated in this work. A comparative study involving high-level features, baseline annotations of multiple modalities and the low-level features is carried out. Optical flow based hidden structure learning of behaviours is strongly discriminatory in predicting a child{\textquoteright}s engagement level in a social interaction. A two-stage approach of discovering the hidden structures using Hidden Conditional Random Fields, followed by learning an SVM-based model on the hidden state marginals is proposed. This is validated by conducting experiments on the Multimodal Dyadic Behaviour Dataset and the results indicate a state of the art classification performance. The insights drawn from this study indicate the robustness of the low-level feature approach towards engagement behaviourmodelling and can be a good substitute in the absence of accurate high-level features.",

keywords = "Social interactions, computer-modeling, hidden-markov-models",

author = "Shyam RAJAGOPALAN and Ramana ORUGANTI and Roland GOECKE and Agata Rozga",

year = "2015",

month = may,

day = "4",

doi = "10.1109/fg.2015.7163129",

language = "English",

volume = "1",

series = "2015 11th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition, FG 2015",

publisher = "IEEE, Institute of Electrical and Electronics Engineers",

pages = "1--8",

editor = "Kevin Bowyer and Ales Leonardo and Jeff Cohn",

booktitle = "2015 11th IEEE International conference and workshop on Automatic face and gesture recognition (FG 2015)",

address = "United States",

note = "11th IEEE International conference and workshop on Automatic face and gesture recognition 2015 ; Conference date: 04-05-2015 Through 08-05-2015",

url = "http://www.fg2015.org/",

}

RAJAGOPALAN, S, ORUGANTI, R, GOECKE, R & Rozga, A 2015, Play with me - Measuring a child's engagement in a social interaction. in K Bowyer, A Leonardo & J Cohn (eds), 2015 11th IEEE International conference and workshop on Automatic face and gesture recognition (FG 2015). vol. 1, 2015 11th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition, FG 2015, IEEE, Institute of Electrical and Electronics Engineers, Ljubljana, Slovenia, pp. 1-8, 11th IEEE International conference and workshop on Automatic face and gesture recognition 2015, Ljubljana, Slovenia, 4/05/15. https://doi.org/10.1109/fg.2015.7163129

Play with me - Measuring a child's engagement in a social interaction. / RAJAGOPALAN, Shyam; ORUGANTI, Ramana; GOECKE, Roland et al.
2015 11th IEEE International conference and workshop on Automatic face and gesture recognition (FG 2015). ed. / Kevin Bowyer; Ales Leonardo; Jeff Cohn. Vol. 1 Ljubljana, Slovenia: IEEE, Institute of Electrical and Electronics Engineers, 2015. p. 1-8 (2015 11th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition, FG 2015).

Research output: A Conference proceeding or a Chapter in Book › Conference contribution › peer-review

TY - GEN

T1 - Play with me - Measuring a child's engagement in a social interaction

AU - RAJAGOPALAN, Shyam

AU - ORUGANTI, Ramana

AU - GOECKE, Roland

AU - Rozga, Agata

PY - 2015/5/4

Y1 - 2015/5/4

N2 - Due to the challenges in automatically observing child behaviour in a social interaction, an automatic extraction of high-level features, such as head poses and hand gestures, is difficult and noisy, leading to an inaccurate model. Hence, the feasibility of using easily obtainable low-level optical flow based features is investigated in this work. A comparative study involving high-level features, baseline annotations of multiple modalities and the low-level features is carried out. Optical flow based hidden structure learning of behaviours is strongly discriminatory in predicting a child’s engagement level in a social interaction. A two-stage approach of discovering the hidden structures using Hidden Conditional Random Fields, followed by learning an SVM-based model on the hidden state marginals is proposed. This is validated by conducting experiments on the Multimodal Dyadic Behaviour Dataset and the results indicate a state of the art classification performance. The insights drawn from this study indicate the robustness of the low-level feature approach towards engagement behaviourmodelling and can be a good substitute in the absence of accurate high-level features.

AB - Due to the challenges in automatically observing child behaviour in a social interaction, an automatic extraction of high-level features, such as head poses and hand gestures, is difficult and noisy, leading to an inaccurate model. Hence, the feasibility of using easily obtainable low-level optical flow based features is investigated in this work. A comparative study involving high-level features, baseline annotations of multiple modalities and the low-level features is carried out. Optical flow based hidden structure learning of behaviours is strongly discriminatory in predicting a child’s engagement level in a social interaction. A two-stage approach of discovering the hidden structures using Hidden Conditional Random Fields, followed by learning an SVM-based model on the hidden state marginals is proposed. This is validated by conducting experiments on the Multimodal Dyadic Behaviour Dataset and the results indicate a state of the art classification performance. The insights drawn from this study indicate the robustness of the low-level feature approach towards engagement behaviourmodelling and can be a good substitute in the absence of accurate high-level features.

KW - Social interactions

KW - computer-modeling

KW - hidden-markov-models

UR - http://www.scopus.com/inward/record.url?scp=84944930963&partnerID=8YFLogxK

U2 - 10.1109/fg.2015.7163129

DO - 10.1109/fg.2015.7163129

M3 - Conference contribution

VL - 1

T3 - 2015 11th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition, FG 2015

SP - 1

EP - 8

BT - 2015 11th IEEE International conference and workshop on Automatic face and gesture recognition (FG 2015)

A2 - Bowyer, Kevin

A2 - Leonardo, Ales

A2 - Cohn, Jeff

PB - IEEE, Institute of Electrical and Electronics Engineers

CY - Ljubljana, Slovenia

T2 - 11th IEEE International conference and workshop on Automatic face and gesture recognition 2015

Y2 - 4 May 2015 through 8 May 2015

ER -

RAJAGOPALAN S, ORUGANTI R, GOECKE R, Rozga A. Play with me - Measuring a child's engagement in a social interaction. In Bowyer K, Leonardo A, Cohn J, editors, 2015 11th IEEE International conference and workshop on Automatic face and gesture recognition (FG 2015). Vol. 1. Ljubljana, Slovenia: IEEE, Institute of Electrical and Electronics Engineers. 2015. p. 1-8. (2015 11th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition, FG 2015). doi: 10.1109/fg.2015.7163129

Play with me - Measuring a child's engagement in a social interaction

Abstract

Publication series

Conference

Access to Document

Other files and links

Fingerprint

Cite this