Abstract
Micro-expression recognition (MER) is a very challenging task since the motion of Micro-expressions (MEs) is subtle, transient and often occurs in tiny regions of face. To build discriminative representation from tiny regions, this paper proposes a novel two stream network, namely Adaptive and Compact Graph Convolutional Network (ACGCN). To be specific, we propose a novel Cheek Included Facial Graph to build more effective structural graph. Then, we propose the Tightly Connected Strategy to adaptively select structural graph to build compact and discriminative facial graph and adjacency matrix. We design the Small Region module to enlarge the interested feature in tiny regions and extract effective feature to build strong and effective node representations. We also adopt the spatial attention to make the network focus on the visual feature of salient regions. Experiments conducted on two micro-expressions datasets (CASME II, SAMM) show our approach outperforms the previous works.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Chattopadhay, A., Sarkar, A., Howlader, P., Balasubramanian, V.N.: Grad-cam++: generalized gradient-based visual explanations for deep convolutional networks. In: 2018 IEEE Winter Conference on Applications of Computer Vision (WACV), pp. 839–847. IEEE (2018)
Davison, A.K., Lansley, C., Costen, N., Tan, K., Yap, M.H.: SAMM: a spontaneous micro-facial movement dataset. IEEE Trans. Affect. Comput. 9(1), 116–129 (2016)
Ekman, P.: Lie catching and microexpressions. Phil. Decept. 1(2), 5 (2009)
Ekman, P., Friesen, W.V.: Facial action coding system. Environ. Psychol. Nonverbal Behav. (1978)
He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778 (2016)
Kazemi, V., Sullivan, J.: One millisecond face alignment with an ensemble of regression trees. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1867–1874 (2014)
Khor, H.Q., See, J., Liong, S.T., Phan, R.C., Lin, W.: Dual-stream shallow networks for facial micro-expression recognition. In: 2019 IEEE International Conference on Image Processing (ICIP), pp. 36–40. IEEE (2019)
Kipf, T.N., Welling, M.: Semi-supervised classification with graph convolutional networks. arXiv preprint arXiv:1609.02907 (2016)
Kumar, A.J.R., Bhanu, B.: Three stream graph attention network using dynamic patch selection for the classification of micro-expressions. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 2476–2485 (2022)
Lei, L., Chen, T., Li, S., Li, J.: Micro-expression recognition based on facial graph representation learning and facial action unit fusion. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 1571–1580 (2021)
Lei, L., Li, J., Chen, T., Li, S.: A novel graph-tcn with a graph structured representation for micro-expression recognition. In: Proceedings of the 28th ACM International Conference on Multimedia, pp. 2237–2245 (2020)
Li, L., Ding, P., Chen, H., Wu, X.: Frequent pattern mining in big social graphs. IEEE Trans. Emerg. Topics Comput. Intell. 6(3), 638–648 (2021)
Liong, S.T., See, J., Wong, K., Phan, R.C.W.: Less is more: micro-expression recognition from video using apex frame. Signal Process. Image Commun. 62, 82–92 (2018)
Liu, Y.J., Zhang, J.K., Yan, W.J., Wang, S.J., Zhao, G., Fu, X.: A main directional mean optical flow feature for spontaneous micro-expression recognition. IEEE Trans. Affect. Comput. 7(4), 299–310 (2015)
Lo, L., Xie, H.X., Shuai, H.H., Cheng, W.H.: MER-GCN: micro-expression recognition based on relation modeling with graph convolutional networks. In: 2020 IEEE Conference on Multimedia Information Processing and Retrieval (MIPR), pp. 79–84. IEEE (2020)
O’sullivan, M., Frank, M.G., Hurley, C.M., Tiwana, J.: Police lie detection accuracy: the effect of lie scenario. Law Hum Behav. 33(6), 530 (2009)
Pan, H., Xie, L., Lv, Z., Li, J., Wang, Z.: Hierarchical support vector machine for facial micro-expression recognition. Multimedia Tools Appl. 79, 31451–31465 (2020)
Pfister, T., Li, X., Zhao, G., Pietikäinen, M.: Differentiating spontaneous from posed facial expressions within a generic facial expression recognition framework. In: 2011 IEEE International Conference on Computer Vision Workshops (ICCV Workshops), pp. 868–875. IEEE (2011)
Pfister, T., Li, X., Zhao, G., Pietikäinen, M.: Recognising spontaneous facial micro-expressions. In: 2011 International Conference on Computer Vision, pp. 1449–1456. IEEE (2011)
Polikovsky, S., Kameda, Y., Ohta, Y.: Facial micro-expressions recognition using high speed camera and 3d-gradient descriptor (2009)
Polikovsky, S., Kameda, Y., Ohta, Y.: Facial micro-expression detection in hi-speed video based on facial action coding system (FACS). IEICE Trans. Inf. Syst. 96(1), 81–92 (2013)
Van Quang, N., Chun, J., Tokuyama, T.: Capsulenet for micro-expression recognition. In: 2019 14th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2019), pp. 1–7. IEEE (2019)
Velickovic, P., et al.: Graph attention networks. STAT 1050(20), 10–48550 (2017)
Verma, M., Vipparthi, S.K., Singh, G.: Affectivenet: affective-motion feature learning for microexpression recognition. IEEE Multimedia 28(1), 17–27 (2020)
Wang, Y., See, J., Phan, R.C.-W., Oh, Y.-H.: LBP with six intersection points: reducing redundant information in LBP-TOP for micro-expression recognition. In: Cremers, D., Reid, I., Saito, H., Yang, M.-H. (eds.) ACCV 2014. LNCS, vol. 9003, pp. 525–537. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-16865-4_34
Wei, J., Peng, W., Lu, G., Li, Y., Yan, J., Zhao, G.: Geometric graph representation with learnable graph structure and adaptive au constraint for micro-expression recognition. arXiv preprint arXiv:2205.00380 (2022)
Wei, M., Zheng, W., Zong, Y., Jiang, X., Lu, C., Liu, J.: A novel micro-expression recognition approach using attention-based magnification-adaptive networks. In: ICASSP 2022-2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 2420–2424. IEEE (2022)
Zhao, G., Pietikainen, M.: Dynamic texture recognition using local binary patterns with an application to facial expressions. IEEE Trans. Pattern Anal. Mach. Intell. 29(6), 915–928 (2007)
Zhao, X., Ma, H., Wang, R.: STA-GCN: spatio-temporal AU graph convolution network for facial micro-expression recognition. In: Ma, H., et al. (eds.) PRCV 2021. LNCS, vol. 13019, pp. 80–91. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-88004-0_7
Zhou, L., Mao, Q., Dong, M.: Objective class-based micro-expression recognition through simultaneous action unit detection and feature aggregation. arXiv preprint arXiv:2012.13148 (2020)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2024 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Ba, R., Li, X., Yang, R., Li, C., Liu, Z. (2024). Adaptive and Compact Graph Convolutional Network for Micro-expression Recognition. In: Liu, Q., et al. Pattern Recognition and Computer Vision. PRCV 2023. Lecture Notes in Computer Science, vol 14433. Springer, Singapore. https://doi.org/10.1007/978-981-99-8546-3_13
Download citation
DOI: https://doi.org/10.1007/978-981-99-8546-3_13
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-99-8545-6
Online ISBN: 978-981-99-8546-3
eBook Packages: Computer ScienceComputer Science (R0)