A Review of Deep Learning-Based Micro-Expression Classification

Authors

  • Yifei Zhang

DOI:

https://doi.org/10.54097/0rn21d50

Keywords:

micro-expression; machine learning; deep learning; CNN; GCN.

Abstract

In daily life, it is important to discern the emotions of others. People always rely on facial expressions and body language to discern the emotional tendencies of others, so that they can choose appropriate ways to communicate and interact with others. Macro expression has a large range of action, while micro expression has a small range of action and is not easy to detect. The former can be falsified while the latter are generated spontaneously. Therefore, analyzing micro-expressions can more accurately determine a person's true emotions, which can then assist criminal investigation or provide psychological therapy. However, even trained individuals find it challenging to correctly identify micro-expressions because of their fleeting and nuanced characteristics, hence, machine learning techniques are frequently employed to handle such tasks. This article looks into deep learning and conventional machine learning algorithms for micro-expression classification. Sentiment analysis methods based on machine learning usually detect facial changes based on texture features and optical flow features. Deep learning methods can be used to complete end-to-end classification through modules such as CNN, GCN and transfer learning. Finally, it was found that to further improve classification accuracy, enhancing the quality of datasets and attempting to apply self-supervised methods to train deep models could be beneficial.

Downloads

Download data is not yet available.

References

MOILANEN A, ZHAO GY, PIETIKÄINEN M. Spotting rapid facial movements from videos using appearance-based feature difference analysis. Washington D. C. , USA: IEEE Press, 2014, 1722-1727.

LIU Y, et al., A Main Directional Mean Optical Flow Feature for Spontaneous Micro-Expression Recognition. IEEE Transactions on Affective Computing, 2016, 7(4): 299-310.

POLIKOVSKY, S., Y. KAMEDA and Y. OHTA, Facial Micro-Expression Detection in Hi-Speed Video Based on Facial Action Coding System (FACS). IEICE Transactions on Information and Systems, 2013, E96.D(1): 81-92.

SHREVE, M., et al. Macro- and micro-expression spotting in long videos using spatio-temporal strain. IEEE, 2011.

YAN W J, LI X, WANG S J, et al. CASMEII: an improved spontaneous micro-expression database and the baseline evaluation. PLoS One, 2014, 9(1): 86041.

Davison, A.K., et al., SAMM: A Spontaneous Micro-Facial Movement Dataset. IEEE Transactions on Affective Computing, 2018, 9(1): 116-129.

PFISTER T,LI X B,ZHAO G Y,et al. Recognising spontaneous facial micro-expressions. Washington D. C. USA: IEEE Press, 2011, 1449-1456.

PIAO LZ, et al., Facial Expression Recognition Using Statistical Shape Analysis. Journal of Ningbo University, 2009, 22(02): 196-201.

XU, F., J. Zhang and J.Z. Wang, Microexpression Identification and Categorization Using a Facial Dynamics Map. IEEE Transactions on Affective Computing, 2017, 8(2): 254-267.

TU L and LIU BY, PCA-based Modification of Dynamics Map Feature Extraction in Micro-Expression Recognition. Communications Technology, 2019, 52(02): 337-342.

LI Y, et al., Progress and challenges in facial action unit detection. Journal of Image and Graphics, 2020, 25(11): 2293-2305.

LI, Y., X. HUANG and G. Zhao, Joint Local and Global Information Learning with Single Apex Frame Detection for Micro-Expression Recognition. IEEE Transactions on Image Processing, 2021, 30: 249-263.

YAO LS, XU GM, et al., Facial Expression Recognition Based on Local Feature Fusion of Convolutional Neural Network. Laser & Optoelectronics Progress, 2020, 57(04): 338-345.

LO, L., et al. MER-GCN: Micro-Expression Recognition Based on Relation Modeling with Graph Convolutional Networks. IEEE, 2020.

ZHAO JH and YANG QX, Micro-expression Recognition Based on AU-GCN and Attention Mechanism. Computer and Modernization, 2023(03): 48-53.

WANG, S., et al., Micro-Expression Recognition Using Color Spaces. IEEE Transactions on Image Processing, 2015, 24(12): 6034-6047.

WANG C Y. Micro-attention for micro-expression recognition. Neurocomputing, 2020, 410: 354-362.

KANG J, LI SY, Transfer Learning Method for Facial Expression Recognition Base on Attention Mechanism. Computer Engineering and Design, 2021, 42(03): 797-804.

Downloads

Published

26-06-2024

How to Cite

Zhang, Y. (2024). A Review of Deep Learning-Based Micro-Expression Classification. Highlights in Science, Engineering and Technology, 103, 109-114. https://doi.org/10.54097/0rn21d50