research-article

Image set-based face recognition using pose estimation with facial landmarks

Authors:

Wenjiao BianAuthors Info & Claims

Multimedia Tools and Applications, Volume 79, Issue 27-28

Pages 19493 - 19507

https://doi.org/10.1007/s11042-019-08408-0

Published: 01 July 2020 Publication History

Abstract

Face recognition (FR) based on image set is an important topic in computer vision. There are numerous approaches that apply pose estimation method for single image face recognition, but few embed pose estimation method into image set-based face recognition. The conventional pose estimation method PnP used for single image needs to be modified to be fit for set-based recognition task. This study presents a method to estimate the poses of image set by applying nonlinear least squares to facial landmarks. Moreover, the distances between every single image in the query set and the ones with similar corresponding poses in the gallery sets are compared. We improve the conventional PnP method by identifying a frontal image of each image set instead of using a fixed 3-D model. Our method is evaluated on the benchmark Honda/UCSD database and YouTube Celebrities database. Experimental results show that our method leads the performance in FR based on image sets compared with other published methods.

References

[1]

Cevikalp H, Triggs B (2010) Face recognition based on image sets. Computer Vision and Pattern Recognition IEEE:2567–2573

[2]

Cootes T (2000) An introduction to active shape models

[3]

De-La-Torre M et al. Adaptive skew-sensitive ensembles for face recognition in video surveillance Pattern Recogn 2015 48 11 3385-3406

[4]

De-La-Torre M et al. Partially-supervised learning from facial trajectories for face recognition in video surveillance Information Fusion 2015 24 3 31-53

[5]

Dewan MAA et al Adaptive appearance model tracking for still-to-video face recognition. Pattern Recogn 49(C, 2016):129–151

[6]

Dorao CA et al. A least squares method for the solution of population balance problems Comput Chem Eng 2006 30 3 535-547

[7]

Edwards GJ, Taylor CJ, Cootes TF (1999) Improving Identification Performance by Integrating Evidence from Sequences. 1 (6): 1486–1486

[8]

Haghighat M, Abdel-Mottaleb M (2017) Low resolution face recognition in surveillance systems using discriminant correlation analysis. The 12th IEEE International Conference on Automatic Face & Gesture Recognition. IEEE 2017

[9]

Harandi MT et al (2011) Graph embedding discriminant analysis on Grassmannian manifolds for improved image set matching. IEEE Conference on Computer Vision & Pattern Recognition IEEE Computer Society

[10]

Hayat M, Bennamoun M, An S (2014) Learning Non-linear Reconstruction Models for Image Set Classification. IEEE Conference on Computer Vision and Pattern Recognition IEEE Computer Society: 1915–1922

[11]

Hu Y, Mian AS, and Owens R Face Recognition Using Sparse Approximated Nearest Points between Image Sets IEEE Transactions on Pattern Analysis & Machine Intelligence 2012 34 10 1992

[12]

Huang L, Lu J, and Tan YP Multi-manifold metric learning for face recognition based on image sets J Vis Commun Image Represent 2014 25 7 1774-1783

[13]

Kim TK, Kittler J, and Cipolla R Discriminative learning and recognition of image set classes using canonical correlations IEEE Transactions on Pattern Analysis & Machine Intelligence 2007 29 6 1005-1018

[14]

Kim M et al (2008) Face tracking and recognition with visual constraints in real-world videos. Computer Vision and Pattern Recognition, 2008. CVPR 2008. IEEE Conference on IEEE, 1–8

[15]

Kløve TLower bounds on the size of spheres of permutations under the Chebychev distanceDes Codes Crypt2011591–3183-19127816081250.05014

[16]

Lee KC et al (2003) Video-based face recognition using probabilistic appearance manifolds. IEEE Computer Society Conference on Computer Vision and Pattern Recognition IEEE Computer Society, 313–320

[17]

Li B, Zheng CH, and Huang DSLocally linear discriminant embedding: An efficient method for face recognitionPattern Recogn200841123813-38211173.68673

[18]

Marquardt DWAn Algorithm for Least-Squares Estimation of Nonlinear ParametersJournal of the Society for Industrial & Applied Mathematics19631124311530710112.10505

[19]

Mian AS, Bennamoun M, and Owens R An efficient multimodal 2D-3D hybrid approach to automatic face recognition IEEE Transactions on Pattern Analysis & Machine Intelligence 2007 29 11 1927-1943

[20]

Milborrow, Stephen, and F. Nicolls. Locating Facial Features with an Extended Active Shape Model. European Conference on Computer Vision Springer-Verlag, (2008):504–513.

[21]

Milborrow S, Nicolls F (2008) Locating Facial Features with an Extended Active Shape Model. European Conference on Computer Vision Springer-Verlag, 504–513

[22]

Pagano C et al. Adaptive ensembles for face recognition in changing video surveillance environments Information Sciences An International Journal 2014 286 11 75-101

[23]

Pal SK and King RA Image enhancement using fuzzy set Electron Lett 1980 16 10 376-378

[24]

Shah SAA, Bennamoun M, and Boussaid F Iterative deep learning for image set based face and object recognition Neurocomputing 2016 174 866-874

[25]

Shah SAA, Bennamoun M, Boussaid F (2016) Iterative deep learning for image set based face and object recognition. Elsevier Science Publishers BV

[26]

Tan X, Chen S, Zhou ZH, et al.Face recognition from a single image per person: a survey[J]Pattern Recogn20063991725-17451096.68732

[27]

Tao D Bayesian tensor approach for 3-D face modeling IEEE Trans Circuits Syst Video Techn 2008 18 10 1397-1410

[28]

Tao D, Guo Y, Li Y, et al. Tensor Rank Preserving Discriminant Analysis for Facial Recognition IEEE Trans Image Process 2017 PP 99 1

[29]

Viola P, Jones MJ, et al. Int J Comput Vis 2004 57 2 137-154

[30]

Wang R, Chen X (2009) Manifold Discriminant Analysis. Computer Vision and Pattern Recognition, 2009. CVPR 2009. IEEE Conference on IEEE, 429–436

[31]

Wang R et al (2008) Manifold-Manifold Distance with application to face recognition based on image set. IEEE Computer Society Conference on Computer Vision and Pattern Recognition DBLP: 1–8

[32]

Wang R et al (2012) Covariance discriminative learning: A natural and efficient approach to image set classification. Computer Vision and Pattern Recognition (CVPR), 2012 IEEE Conference on IEEE

[33]

Yamaguchi O, Fukui K, Maeda K (1998) Face Recognition Using Temporal Image Sequence. IEEE International Conference on Automatic Face and Gesture Recognition, 1998. Proceedings IEEE, 318–323

[34]

Yang M, Liu W, Shen L (2015) Joint regularized nearest points for image set based face recognition. IEEE International Conference and Workshops on Automatic Face and Gesture Recognition IEEE, 1–7

[35]

Yang W, Minoh M, Mukunoki M (2013) Collaboratively Regularized Nearest Points for Set Based Recognition. British Machine Vision Conference:134.1–134.11

[36]

Yang M et al (2013) Face recognition based on regularized nearest points between image sets. IEEE International Conference and Workshops on Automatic Face and Gesture Recognition IEEE, 1–7

[37]

Zeng, QS, Lai JH, Wang CD (2014) Multi-local model image set matching based on domain description. Elsevier Science Inc

[38]

Zhang J, Yan K, Zhen-Yu HE et al (2014) A collaborative linear discriminative representation classification based method for face recognition. International Conference on Artificial Intelligence and Software Engineering (AISE 2014)

[39]

Zhao ZQ, Xu ST, Liu D et al (2018) A review of image set classification. Neurocomputing

[40]

Zhao W et al.Face recognition:a literature surveyACM Comput Surv2003354399-4581435164

[41]

Zhou S, Krueger V, Chellappa R (2002) Face recognition from video: a CONDENSATION approach. IEEE International Conference on Automatic Face and Gesture Recognition. Proceedings IEEE, 2002:221

[42]

Zhu L, Shen J, Xie L, Cheng Z (2017) Unsupervised topic hypergraph hashing for efficient mobile image retrieval. IEEE Trans Cybern 47(11):3941–3954

[43]

Zhu P et al. Image Set-Based Collaborative Representation for Face Recognition IEEE Transactions on Information Forensics & Security 2014 9 7 1120-1132

Cited By

Zhang HYang JDong XLv XJia WJin ZLi X(2023)A Video Face Recognition Leveraging Temporal Information Based on Vision TransformerPattern Recognition and Computer Vision10.1007/978-981-99-8469-5_3(29-43)Online publication date: 13-Oct-2023
https://dl.acm.org/doi/10.1007/978-981-99-8469-5_3

Recommendations

Frontal face synthesis based on multiple pose-variant images for face recognition
ICB'07: Proceedings of the 2007 international conference on Advances in Biometrics

Pose variance remains a challenging problem for face recognition. In this paper, a stereoscopic synthesis method for generating a frontal face image is proposed to improve the performance of automatic face recognition system. Through this method, a ...
Pose‐invariant face recognition based on matching the occlusion free regions aligned by 3D generic model

Face recognition systems perform accurately in a controlled environment, but an unconstrained environment dramatically degrades their performance. In this study, a novel pose‐invariant face recognition system is proposed based on the occlusion free ...
Pose-invariant face recognition using facial landmarks and Weber local descriptor

Face recognition across pose is a popular issue in biometrics. Facial rotations caused by pose dramatically enlarge the intra-class variations, which considerably obstructs the performance of the face recognition algorithms. It is advisable to extract ...

Comments

Information & Contributors

Information

Published In

cover image Multimedia Tools and Applications

Multimedia Tools and Applications Volume 79, Issue 27-28

Jul 2020

1552 pages

ISSN:1380-7501

Issue’s Table of Contents

© Springer Science+Business Media, LLC, part of Springer Nature 2019.

Publisher

Kluwer Academic Publishers

United States

Publication History

Published: 01 July 2020

Accepted: 01 November 2019

Revision received: 05 October 2019

Received: 24 November 2018

Author Tags

Qualifiers

Research-article

Funding Sources

National Natural Science Foundation of China

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

1
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 22 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Zhang HYang JDong XLv XJia WJin ZLi X(2023)A Video Face Recognition Leveraging Temporal Information Based on Vision TransformerPattern Recognition and Computer Vision10.1007/978-981-99-8469-5_3(29-43)Online publication date: 13-Oct-2023
https://dl.acm.org/doi/10.1007/978-981-99-8469-5_3

View Options

View options

Figures

Tables

Media

View Issue’s Table of Contents