research-article

Isolating Uncertainty of the Face Expression Recognition with the Meta-Learning Supervisor Neural Network

Authors:

Stanislav Selitskiy,

Nikolaos Christou,

Natalya SelitskayaAuthors Info & Claims

AIVR 2021: 2021 5th International Conference on Artificial Intelligence and Virtual Reality (AIVR)

Pages 104 - 112

https://doi.org/10.1145/3480433.3480447

Published: 08 November 2021 Publication History

Abstract

We investigate whether the well-known poor performance of the head-on usage of the convolutional neural networks for the facial expression recognition task may be improved in terms of reducing the false positive and false negative errors. An uncertainty isolating technique is used that introduces an additional “unknown” class. A self-attention supervisor artificial neural network is used to “learn about learning” of the underlying convolutional neural networks, in particular, to learn patterns of the underlying neural network parameters that accompany wrong or correct verdicts. A novel data set containing artistic makeup and occlusions images is used to aggravate the problem of the training data not representing the test data distribution.

References

[1]

Face Matching Data Set |Biometric Data| CyberExtruder, Dec 2019. [Online; accessed 8/12/2019].

[2]

Dario Amodei, Chris Olah, Jacob Steinhardt, Paul Christiano, John Schulman, and Dan Man ́e. Concrete problems in ai safety, 2016.

[3]

Marcin Andrychowicz, Misha Denil, Sergio Gomez, Matthew W. Hoffman, David Pfau, Tom Schaul, Brendan Shillingford, and Nando de Freitas. Learning to learn by gradient descent by gradient descent, 2016.

[4]

James Bergstra, R ́emi Bardenet, Yoshua Bengio, and Bal ́azsK ́egl. Algorithms for hyper-parameter optimization. In J. Shawe-Taylor, R. Zemel, P. Bartlett, F. Pereira, and K. Q.Weinberger, editors, Advances in Neural Information Processing Systems, volume 24. Curran Associates, Inc., 2011.

[5]

Stefano Berretti, Alberto Del Bimbo, Pietro Pala, BoulbabaBen Amor, and Daoudi Mohamed. A Set of Selected SIFT Features for 3D Facial Expression Recognition. In20th International Conference on Pattern Recognition, pages 4125– 4128, Istanbul, Turkey, Aug. 2010.

Digital Library

[6]

W. Briggs.Uncertainty: The Soul of Modeling, Probability& Statistics. Springer International Publishing, 2016.

[7]

John T Cacioppo, Gary G Berntson, Jeff T Larsen, Kirsten M Poehlmann, Tiffany A Ito, The psychophysiology of emotion. Handbook of emotions, 2:173–191, 2000.

[8]

Cunjian Chen, Antitza Dantcheva, and Arun Ross. Automatic facial makeup detection with application in face recog-nition.2013 International Conference on Biometrics (ICB), pages 1–8, 2013.

[9]

C. Chen, A. Dantcheva, T. Swearingen, and A. Ross. Spoofing faces using makeup: An investigative study. In 2017 IEEE International Conference on Identity, Security and Behavior Analysis (ISBA), pages 1–8, Feb. 2017.

[10]

A. Colombo, C. Cusano, and R. Schettini. Umb-db: A database of partially occluded 3d faces. In2011 IEEE Inter-national Conference on Computer Vision Workshops (ICCVWorkshops), pages 2113–2119, Nov. 2011.

[11]

Tee Connie, Mundher Al-Shabi, Wooi Ping Cheah, and Michael Goh. Facial expression recognition using a hybrid cnn–sift aggregator. Multi-disciplinary Trends in Artificial Intelligence, page 139–149, 2017.

[12]

A. Dantcheva, C. Chen, and A. Ross. Can facial cosmetics affect the matching accuracy of face recognition systems? In IEEE Fifth International Conference on Biometrics: Theory, Applications and Systems (BTAS), pages 391–398, Sep. 2012.

[13]

M. Eckert, N. Kose, and J. Dugelay. Facial cosmetics database and impact analysis on automatic face recognition. In IEEE 15th International Workshop on Multimedia Signal Processing (MMSP), pages 434–439, Sep. 2013.

[14]

P Ekman and W V Friesen. Constants across cultures in the face and emotion. J Pers Soc Psychol, 17(2):124–129, Feb. 1971.

[15]

Ranran Feng and Balakrishnan Prabhakaran. Facilitating fashion camouflage art. In Proceedings of the 21st ACM International Conference on Multimedia, MM ’13, pages 793–802, New York, NY, USA, 2013. ACM

Digital Library

[16]

Chelsea Finn, Pieter Abbeel, and Sergey Levine. Model-agnostic meta-learning for fast adaptation of deep networks, 2017.

[17]

Yarin Gal and Zoubin Ghahramani. Dropout as a Bayesian approximation: Representing model uncertainty in deep learning, 2016.

[18]

Z Ghahramani. Probabilistic machine learning and artificial intelligence. Nature, 521(7553):452–459, May 2015.

[19]

Alex Graves. Practical variational inference for neural net-works. In J. Shawe-Taylor, R. Zemel, P. Bartlett, F. Pereira, and K. Q. Weinberger, editors, Advance in Neural Information Processing Systems, volume 24, pages 2348–2356.Curran Associates, Inc., 2011.

[20]

Cornelius T Gross and Newton Sabino Canteras. The many paths to fear. Nat Rev Neurosci, 13(9):651–658, Sep. 2012.

[21]

Paul Halmos. Naive Set Theory. Van Nostrand, 1960.Reprinted by Springer-Verlag, Undergraduate Texts in Mathematics, 1974.

[22]

Geoffrey E. Hinton and R. Neal. Bayesian learning for neural networks. 1995.

Digital Library

[23]

Gary B. Huang, Marwan Mattar, Tamara Berg, and Eric Learned-Miller. Labeled Faces in the Wild: A Database for Studying Face Recognition in Unconstrained Environments. In Workshop on Faces in ’Real-Life’ Images: Detection, Alignment, and Recognition, France, 2008.

[24]

Shan Jia, Xin Li, Chuanbo Hu, Guodong Guo, and Zhengquan Xu. 3d face anti-spoofing with factorized bilinear coding, 2020.

[25]

Alex Kendall and Yarin Gal. What uncertainties do we need in Bayesian deep learning for computer vision? In I. Guyon, U. V. Luxburg, S. Bengio, H. Wallach, R. Fergus, S. Vishwanathan, and R. Garnett, editors, Advances in Neural Information Processing Systems, volume 30, pages 5574–5584.Curran Associates, Inc., 2017.

[26]

A. Khodabakhsh and C. Busch. A generalizable deep fake detector based on neural conditional distribution modelling. In2020 International Conference of the Biometrics Special Interest Group (BIOSIG), pages 1–5, 2020.

[27]

Bo-Kyeong Kim, Jihyeon Roh, Suh-Yeon Dong, and S. Lee. Hierarchical committee of deep convolutional neural networks for robust facial expression recognition. Journal on Multimodal User Interfaces, 10:173–189, 2015.

[28]

Zeshan Kurd and Tim Kelly. Establishing safety criteria forartificial neural networks. In Palade V., Howlett R.J., JainL. (eds) Knowledge-Based Intelligent Information and Engineering Systems, volume 2773. Springer, Berlin, Heidelberg, 2003.

[29]

V. Kushwaha, M. Singh, R. Singh, M. Vatsa, N. Ratha,and R. Chellappa. Disguised faces in the wild. In 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), pages 1–18, June 2018.

[30]

Brenden M. Lake, Tomer D. Ullman, Joshua B. Tenenbaum, and Samuel J. Gershman. Building machines that learn and think like people, 2016.

[31]

Balaji Lakshminarayanan, Alexander Pritzel, and Charles Blundell. Simple and scalable predictive uncertainty estimation using deep ensembles, 2017.

[32]

J. Li and E. Lam. Facial expression recognition using deep neural networks. 2015 IEEE International Conference on Imaging Systems and Techniques (IST), pages 1–6, 2015.

[33]

Shan Li and Weihong Deng. Deep facial expression recognition: A survey. IEEE Transactions on Affective Computing, page 1–1, 2020.

[34]

Mengyi Liu, Shaoxin Li, Shiguang Shan, and Xilin Chen. Au-inspired deep networks for facial expression feature learning. Neurocomputing, 159:126–136, 2015.

Digital Library

[35]

X. Liu, X. Wang, and S. Matwin. Interpretable deep convolutional neural networks via meta-learning. In2018 International Joint Conference on Neural Networks (IJCNN), pages 1–9, 2018.

[36]

Andr Teixeira Lopes, Edilson de Aguiar, Alberto F. De Souza, and Thiago Oliveira-Santos. Facial expression recognition with convolutional neural networks. Pattern Recogn., 61(C):610–628, Jan. 2017.

[37]

D. J. C. MacKay. A practical Bayesian framework for back-propagation networks. Neural Computation, 4(3):448–472, 1992.

Digital Library

[38]

Hadi Mansourifar and Weidong Shi. One-shot gan generated fake face detection, 2020.

[39]

Aleix Martinez and Robert Benavente. The ar face database. Technical Report 24, Computer Vision Center, Bellatera, Jun. 1998.

[40]

Rui Min, Abdenour Hadid, and Jean-Luc Dugelay. Improving the recognition of faces occluded by facial accessories. In FG 2011, 9th IEEE Conference on Automatic Face and Gesture Recognition, March 21-25, 2011.

[41]

Shervin Minaee and Amirali Abdolrashidi. Deep-emotion: Facial expression recognition using attentional convolutional network, 2019.

[42]

S. Mitra, Mikhail I. Gofman, G. Parsons, and J. Peissig. Facial asymmetry versus facial makeup.2018 IEEE 8th Annual Computing and Communication Workshop and Conference (CCWC), pages 197–203, 2018.

[43]

Ali Mollahosseini, David Chan, and Mohammad H. Mahoor. Going deeper in facial expression recognition using deep neural networks. 2016 IEEE Winter Conference on Applications of Computer Vision (WACV), Mar 2016.

[44]

Radford M Neal. Bayesian learning for neural networks, volume 118. Springer Science & Business Media, 2012.

[45]

Hongwei Ng, V. D. Nguyen, V. Vonikakis, and S. Winkler. Deep learning for emotion recognition on small datasets using transfer learning. Proceedings of the 2015 ACM on International Conference on Multimodal Interaction, 2015.

Digital Library

[46]

Alex Nichol, Joshua Achiam, and John Schulman. On first-order meta-learning algorithms, 2018.

[47]

Diah Anggraeni Pitaloka, Ajeng Wulandari, T. Basaruddin, and Dewi Yanti Liliana. Enhancing cnn with preprocessing stage in automatic emotion recognition. Procedia Computer Science, 116:523–529, 2017. Discovery and innovation of computer science technology in artificial intelligence era: The 2nd International Conference on Computer Science and Computational Intelligence (ICCSCI 2017)

Digital Library

[48]

Janis Postels, Francesco Ferroni, Huseyin Coskun, Nassir Navab, and Federico Tombari. Sampling-free epistemic un-certainty estimation using approximated variance propagation, 2019.

[49]

Raju Ram, Sabine M ̈uller, Franz-Josef Pfreundt, Nicolas R. Gauger, and Janis Keuper. Scalable hyperparameter optimization with lazy gaussian processes, 2020.

[50]

Adam Santoro, Sergey Bartunov, Matthew Botvinick, DaanWierstra, and Timothy Lillicrap. Meta-learning with memory-augmented neural networks. In Maria Florina Balcan and Kilian Q. Weinberger, editors, Proceedings of The 33rd International Conference on Machine Learning, volume 48 of Proceedings of Machine Learning Research, pages 1842–1850, New York, New York, USA, 20–22 Jun. 2016. PMLR.

[51]

S. Setty, M. Husain, and P. Beham etc. Indian movie face database: A benchmark for face recognition under wide variations. In Fourth National Conference on Computer Vision, Pattern Recognition, Image Processing and Graphics, pages1–5, 2013.

[52]

Caifeng Shan, Shaogang Gong, and Peter W. McOwan. Facial expression recognition based on local binary patterns: A comprehensive study. Image and Vision Computing, 27(6):803–816, 2009.

Digital Library

[53]

Pratt L. Thrun S. Learning To Learn. Springer, Boston, MA, 1998.

[54]

Joaquin Vanschoren. Meta-learning: A survey, ArXiv abs/1810.03548, 2018.

[55]

Shuyang Wang and Yun Fu. Face behind makeup. In Proceedings of the Thirtieth AAAI Conference on Artificial Intelligence, AAAI’16, page 58–64. AAAI Press, 2016.

[56]

J. Whitehill and C. W. Omlin. Haar features for facs au recognition. In7th International Conference on Automatic Face and Gesture Recognition (FGR06), pages 5, 2006.

Digital Library

[57]

Oliver Willers, Sebastian Sudholt, Shervin Raafatnia, and Stephanie Abrecht. Safety concerns and mitigation approaches regarding the use of deep learning in safety-critical perception tasks, 2020.

Digital Library

[58]

B. F. Wu and C. H. Lin. Adaptive feature mapping for customizing deep learning based facial expression recognition model. IEEE Access, 6:12451–12461, 2018.

[59]

Tianchen Zhao, Xiang Xu, Mingze Xu, Hui Ding, Yuanjun Xiong, and Wei Xia. Learning to recognize patch-wise consistency for deep fake detection, 2020.

Cited By

Selitskiy S(2024)Explicit Model Memorisation to Fight Forgetting in Time-series PredictionSoutheastCon 202410.1109/SoutheastCon52093.2024.10500223(660-667)Online publication date: 15-Mar-2024
https://doi.org/10.1109/SoutheastCon52093.2024.10500223
Selitskiy S(2023)Elements of Continuous Reassessment and Uncertainty Self-awareness: A Narrow Implementation for Face and Facial Expression RecognitionAdvanced Intelligent Virtual Reality Technologies10.1007/978-981-19-7742-8_5(61-71)Online publication date: 20-Jan-2023
https://doi.org/10.1007/978-981-19-7742-8_5

Index Terms

Isolating Uncertainty of the Face Expression Recognition with the Meta-Learning Supervisor Neural Network
1. Computing methodologies

Index terms have been assigned to the content through auto-classification.

Recommendations

Facial Expression Recognition Using Neural Network Trained with Zernike Moments
ICAIET '14: Proceedings of the 2014 4th International Conference on Artificial Intelligence with Applications in Engineering and Technology

Neural network classifying method is used in this work to perform facial expression recognition. The processed expressions were the six most pertinent facial expressions and the neutral one. This operation was implemented in three steps. First, a neural ...
Hybrid-boost learning for multi-pose face detection and facial expression recognition

This paper proposes a hybrid-boost learning algorithm for multi-pose face detection and facial expression recognition. To speed-up the detection process, the system searches the entire frame for the potential face regions by using skin color detection ...
Face Expression Recognition Using Artificial Neural Network (ANN) Model Back Propagation
ICASI'18: Proceedings of the Joint Workshop KO2PI and the 1st International Conference on Advance & Scientific Innovation

Humans can detect a facial expressions of any person easier, because its naturally directly recognized, but its very difficult to do by machine or computer. There are 3 main stages when designing a facial expression recognition system, that is, face ...

Comments

Information & Contributors

Information

Published In

cover image ACM Other conferences

AIVR 2021: 2021 5th International Conference on Artificial Intelligence and Virtual Reality (AIVR)

July 2021

134 pages

ISBN:9781450384148

DOI:10.1145/3480433

Copyright © 2021 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 08 November 2021

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed limited

Conference

AIVR 2021

AIVR 2021: 2021 the 5th International Conference on Artificial Intelligence and Virtual Reality

July 23 - 25, 2021

Kumamoto, Japan

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

2
Total Citations
View Citations
43
Total Downloads

Downloads (Last 12 months)13
Downloads (Last 6 weeks)0

Reflects downloads up to 01 Sep 2024

Other Metrics

View Author Metrics

Citations

Cited By

Selitskiy S(2024)Explicit Model Memorisation to Fight Forgetting in Time-series PredictionSoutheastCon 202410.1109/SoutheastCon52093.2024.10500223(660-667)Online publication date: 15-Mar-2024
https://doi.org/10.1109/SoutheastCon52093.2024.10500223
Selitskiy S(2023)Elements of Continuous Reassessment and Uncertainty Self-awareness: A Narrow Implementation for Face and Facial Expression RecognitionAdvanced Intelligent Virtual Reality Technologies10.1007/978-981-19-7742-8_5(61-71)Online publication date: 20-Jan-2023
https://doi.org/10.1007/978-981-19-7742-8_5

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Media

Figures

Other

Tables

View Table of Contents