Micro-Expression Recognition Based on Video Motion Magnification and Pre-trained Neural Network

Keira Bai; Roland Goecke; Damith Herath

doi:10.1109/ICIP42928.2021.9506793

Micro-Expression Recognition Based on Video Motion Magnification and Pre-trained Neural Network

Keira Bai, Roland Goecke, Damith Herath

Information Systems

Research output: A Conference proceeding or a Chapter in Book › Conference contribution › peer-review

7 Citations (Scopus)

Abstract

This paper investigates the effects of using video motion magnification methods based on amplitude and phase, respectively, to amplify small facial movements. We hypothesise that this approach will assist in the micro-expression recognition task. To this end, we apply the pre-trained VGGFace2 model with its excellent facial feature capturing ability to transfer learn the magnified micro-expression movement, then encode the spatial information and decode the spatial and temporal information by Bi-LSTM model. Moreover, Grad-CAM is utilised to map the model and visually explain the operating mechanism of the spatio-temporal network. Experiments on the SMIC database confirm that the proposed framework significantly improves the micro-expression recognition rate compared to without video magnification (baseline) and other state-of-the-art methods.

Original language	English
Title of host publication	2021 IEEE International Conference on Image Processing, ICIP 2021 - Proceedings
Editors	Saif alZahir, Fabrice Labeau, Kenrick Mock
Place of Publication	United States
Publisher	IEEE, Institute of Electrical and Electronics Engineers
Pages	549-553
Number of pages	5
ISBN (Electronic)	9781665441155
ISBN (Print)	9781665431026
DOIs	https://doi.org/10.1109/ICIP42928.2021.9506793
Publication status	Published - 23 Aug 2021
Event	28th IEEE International Conference on Image Processing - Denaʼina Civic and Convention Center, Anchorage, United States Duration: 19 Sept 2021 → 22 Sept 2021 https://2021.ieeeicip.org/

Publication series

Name	Proceedings - International Conference on Image Processing, ICIP
Volume	2021-September
ISSN (Print)	1522-4880

Conference

Conference	28th IEEE International Conference on Image Processing
Abbreviated title	ICIP 2021
Country/Territory	United States
City	Anchorage
Period	19/09/21 → 22/09/21
Internet address	https://2021.ieeeicip.org/

Access to Document

10.1109/ICIP42928.2021.9506793

Cite this

Bai, K., Goecke, R., & Herath, D. (2021). Micro-Expression Recognition Based on Video Motion Magnification and Pre-trained Neural Network. In S. alZahir, F. Labeau, & K. Mock (Eds.), 2021 IEEE International Conference on Image Processing, ICIP 2021 - Proceedings (pp. 549-553). (Proceedings - International Conference on Image Processing, ICIP; Vol. 2021-September). IEEE, Institute of Electrical and Electronics Engineers. https://doi.org/10.1109/ICIP42928.2021.9506793

Bai, Keira ; Goecke, Roland ; Herath, Damith. / Micro-Expression Recognition Based on Video Motion Magnification and Pre-trained Neural Network. 2021 IEEE International Conference on Image Processing, ICIP 2021 - Proceedings. editor / Saif alZahir ; Fabrice Labeau ; Kenrick Mock. United States : IEEE, Institute of Electrical and Electronics Engineers, 2021. pp. 549-553 (Proceedings - International Conference on Image Processing, ICIP).

@inproceedings{8712837cb76f423ab4a2792a1ac99332,

title = "Micro-Expression Recognition Based on Video Motion Magnification and Pre-trained Neural Network",

abstract = "This paper investigates the effects of using video motion magnification methods based on amplitude and phase, respectively, to amplify small facial movements. We hypothesise that this approach will assist in the micro-expression recognition task. To this end, we apply the pre-trained VGGFace2 model with its excellent facial feature capturing ability to transfer learn the magnified micro-expression movement, then encode the spatial information and decode the spatial and temporal information by Bi-LSTM model. Moreover, Grad-CAM is utilised to map the model and visually explain the operating mechanism of the spatio-temporal network. Experiments on the SMIC database confirm that the proposed framework significantly improves the micro-expression recognition rate compared to without video magnification (baseline) and other state-of-the-art methods.",

keywords = "Micro-expression recognition, Motion magnification, Spatio-temporal network",

author = "Keira Bai and Roland Goecke and Damith Herath",

note = "Funding Information: This research was supported partially by the Australian Government through the Australian Research Council{\textquoteright}s Discovery Projects funding scheme (project DP190101294). Publisher Copyright: {\textcopyright} 2021 IEEE.; 28th IEEE International Conference on Image Processing, ICIP 2021 ; Conference date: 19-09-2021 Through 22-09-2021",

year = "2021",

month = aug,

day = "23",

doi = "10.1109/ICIP42928.2021.9506793",

language = "English",

isbn = "9781665431026",

series = "Proceedings - International Conference on Image Processing, ICIP",

publisher = "IEEE, Institute of Electrical and Electronics Engineers",

pages = "549--553",

editor = "Saif alZahir and Fabrice Labeau and Kenrick Mock",

booktitle = "2021 IEEE International Conference on Image Processing, ICIP 2021 - Proceedings",

address = "United States",

url = "https://2021.ieeeicip.org/",

}

Bai, K, Goecke, R & Herath, D 2021, Micro-Expression Recognition Based on Video Motion Magnification and Pre-trained Neural Network. in S alZahir, F Labeau & K Mock (eds), 2021 IEEE International Conference on Image Processing, ICIP 2021 - Proceedings. Proceedings - International Conference on Image Processing, ICIP, vol. 2021-September, IEEE, Institute of Electrical and Electronics Engineers, United States, pp. 549-553, 28th IEEE International Conference on Image Processing, Anchorage, Alaska, United States, 19/09/21. https://doi.org/10.1109/ICIP42928.2021.9506793

Micro-Expression Recognition Based on Video Motion Magnification and Pre-trained Neural Network. / Bai, Keira; Goecke, Roland; Herath, Damith.
2021 IEEE International Conference on Image Processing, ICIP 2021 - Proceedings. ed. / Saif alZahir; Fabrice Labeau; Kenrick Mock. United States: IEEE, Institute of Electrical and Electronics Engineers, 2021. p. 549-553 (Proceedings - International Conference on Image Processing, ICIP; Vol. 2021-September).

Research output: A Conference proceeding or a Chapter in Book › Conference contribution › peer-review

TY - GEN

T1 - Micro-Expression Recognition Based on Video Motion Magnification and Pre-trained Neural Network

AU - Bai, Keira

AU - Goecke, Roland

AU - Herath, Damith

N1 - Funding Information: This research was supported partially by the Australian Government through the Australian Research Council’s Discovery Projects funding scheme (project DP190101294). Publisher Copyright: © 2021 IEEE.

PY - 2021/8/23

Y1 - 2021/8/23

N2 - This paper investigates the effects of using video motion magnification methods based on amplitude and phase, respectively, to amplify small facial movements. We hypothesise that this approach will assist in the micro-expression recognition task. To this end, we apply the pre-trained VGGFace2 model with its excellent facial feature capturing ability to transfer learn the magnified micro-expression movement, then encode the spatial information and decode the spatial and temporal information by Bi-LSTM model. Moreover, Grad-CAM is utilised to map the model and visually explain the operating mechanism of the spatio-temporal network. Experiments on the SMIC database confirm that the proposed framework significantly improves the micro-expression recognition rate compared to without video magnification (baseline) and other state-of-the-art methods.

AB - This paper investigates the effects of using video motion magnification methods based on amplitude and phase, respectively, to amplify small facial movements. We hypothesise that this approach will assist in the micro-expression recognition task. To this end, we apply the pre-trained VGGFace2 model with its excellent facial feature capturing ability to transfer learn the magnified micro-expression movement, then encode the spatial information and decode the spatial and temporal information by Bi-LSTM model. Moreover, Grad-CAM is utilised to map the model and visually explain the operating mechanism of the spatio-temporal network. Experiments on the SMIC database confirm that the proposed framework significantly improves the micro-expression recognition rate compared to without video magnification (baseline) and other state-of-the-art methods.

KW - Micro-expression recognition

KW - Motion magnification

KW - Spatio-temporal network

UR - http://www.scopus.com/inward/record.url?scp=85125580427&partnerID=8YFLogxK

U2 - 10.1109/ICIP42928.2021.9506793

DO - 10.1109/ICIP42928.2021.9506793

M3 - Conference contribution

SN - 9781665431026

T3 - Proceedings - International Conference on Image Processing, ICIP

SP - 549

EP - 553

BT - 2021 IEEE International Conference on Image Processing, ICIP 2021 - Proceedings

A2 - alZahir, Saif

A2 - Labeau, Fabrice

A2 - Mock, Kenrick

PB - IEEE, Institute of Electrical and Electronics Engineers

CY - United States

T2 - 28th IEEE International Conference on Image Processing

Y2 - 19 September 2021 through 22 September 2021

ER -

Bai K, Goecke R, Herath D. Micro-Expression Recognition Based on Video Motion Magnification and Pre-trained Neural Network. In alZahir S, Labeau F, Mock K, editors, 2021 IEEE International Conference on Image Processing, ICIP 2021 - Proceedings. United States: IEEE, Institute of Electrical and Electronics Engineers. 2021. p. 549-553. (Proceedings - International Conference on Image Processing, ICIP). doi: 10.1109/ICIP42928.2021.9506793

Micro-Expression Recognition Based on Video Motion Magnification and Pre-trained Neural Network

Abstract

Publication series

Conference

Access to Document

Other files and links

Fingerprint

Cite this