Investigating LSTM for Micro-Expression Recognition

Mengjiong Bai, Roland Goecke

Research output: A Conference proceeding or a Chapter in BookConference contributionpeer-review

10 Citations (Scopus)


This study investigates the utility of Long Short-Term Memory (LSTM) networks for modelling spatial-temporal patterns for micro-expression recognition (MER). Micro-expressions are involuntary, short facial expressions, often of low intensity. RNNs have attracted a lot of attention in recent years for modelling temporal sequences. The RNN-LSTM combination to be highly effective results in many application areas. The proposed method combines the recent VGGFace2 model, basically a ResNet-50 CNN trained on the VGGFace2 dataset, with uni-directional and bi-directional LSTM to explore different ways modelling spatial-temporal facial patterns for MER. The Grad-CAM heat map visualisation is used in the training stages to determine the most appropriate layer of the VGGFace2 model for retraining. Experiments are conducted with pure VGGFace2, VGGFace2 + uni-directional LSTM, and VGGFace2 + Bi-directional LSTM on the SMIC database using 5-fold cross-validation.
Original languageEnglish
Title of host publicationICMI '20 Companion
Subtitle of host publicationCompanion Publication of the 2020 International Conference on Multimodal Interaction
EditorsKhiet Truong, Dirk Heylen, Mary Czerwinski
Place of PublicationNetherlands
PublisherAssociation for Computing Machinery (ACM)
Number of pages5
ISBN (Print)9781450380027
Publication statusPublished - 25 Oct 2020
Event22nd ACM International Conference on Multimodal Interaction - Utrecht, Netherlands
Duration: 25 Oct 202029 Oct 2020


Conference22nd ACM International Conference on Multimodal Interaction
Abbreviated titleICMI'20
Internet address


Dive into the research topics of 'Investigating LSTM for Micro-Expression Recognition'. Together they form a unique fingerprint.

Cite this