article

Face Alignment by Explicit Shape Regression

Authors:

Jian SunAuthors Info & Claims

International Journal of Computer Vision, Volume 107, Issue 2

Pages 177 - 190

https://doi.org/10.1007/s11263-013-0667-3

Published: 01 April 2014 Publication History

Abstract

We present a very efficient, highly accurate, "Explicit Shape Regression" approach for face alignment. Unlike previous regression-based approaches, we directly learn a vectorial regression function to infer the whole facial shape (a set of facial landmarks) from the image and explicitly minimize the alignment errors over the training data. The inherent shape constraint is naturally encoded into the regressor in a cascaded learning framework and applied from coarse to fine during the test, without using a fixed parametric shape model as in most previous methods. To make the regression more effective and efficient, we design a two-level boosted regression, shape indexed features and a correlation-based feature selection method. This combination enables us to learn accurate models from large training data in a short time (20 min for 2,000 training images), and run regression extremely fast in test (15 ms for a 87 landmarks shape). Experiments on challenging data show that our approach significantly outperforms the state-of-the-art in terms of both accuracy and efficiency.

References

[1]

Belhumeur, P., Jacobs, D., Kriegman, D., & Kumar, N. (2011). Localizing parts of faces using a concensus of exemplars. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[2]

Bingham, E., & Mannila, H. (2001). Random projection in dimensionality reduction: Applications to image and text data. In ACMSIGKDD Conference on Knowledge Discovery and Data Mining (KDD).

[3]

Cootes, T., Edwards, G., & Taylor, C. (2001). Active appearance models. IEEE Transactions on Pattern Analysis and Machine Intelligence, 23(6), 681-685.

Digital Library

[4]

Cootes, T., Taylor, C., Cooper, D., Graham, J., et al. (1995). Active shape models-their training and application. Computer Vision and Image Understanding, 61(1), 38-59.

Digital Library

[5]

Cristinacce, D., & Cootes, T. (2006). Feature detection and tracking with constrained local models. In British Machine Vision Conference (BMVC).

[6]

Cristinacce, D., & Cootes, T. (2007). Boosted regression active shape models. In British Machine Vision Conference (BMVC).

[7]

Dollar, P., Welinder, P., & Perona, P. (2010). Cascaded pose regression. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[8]

Duffy, N., & Helmbold, D. P. (2002). Boosting methods for regression. Machine Learning, 47(2-3), 153-200.

Digital Library

[9]

Elad, M., & Aharon, M. (2006). Image denoising via sparse and redundant representations over learned dictionaries. IEEE Transactions on Image Processing, 15(12), 3736-3745.

Digital Library

[10]

Friedman, J. H. (2001). Greedy function approximation: A gradient boosting machine. The Annals of Statistics, 29(5), 1189-1232.

[11]

Huang, G., Mattar, M., Berg, T., Learned-Miller, E. et al. (2008) Labeled faces in the wild: A database for studying face recognition in unconstrained environments. In Workshop on Faces in'Real-Life'Images: Detection, Alignment, and Recognition.

[12]

Jesorsky, O., Kirchberg, K. J., & Frischholz, R. W. (2001). Robust face detection using the hausdorff distance (pp. 90-95). New York: Springer.

[13]

Jolliffe, I. (2005). Principal component analysis. Wiley Online Library.

[14]

Le, V., Brandt, J., Lin, Z., Bourdev, L., & Huang, T. (2012). Interactive facial feature localization. In European Conference on Computer Vision.

[15]

Liang, L., Xiao, R., Wen, F., & Sun, J. (2008). Face alignment via component-based discriminative search. In European Conference on Computer Vision (ECCV).

[16]

Matthews, I., & Baker, S. (2004). Active appearance models revisited. International Journal of Computer Vision, 60(2), 135-164.

Digital Library

[17]

Milborrow, S., & Nicolls, F. (2008). Locating facial features with an extended active shape model. In European Conference on Computer Vision (ECCV).

[18]

Ozuysal, M., Calonder, M., Lepetit, V., & Fua, P. (2010). Fast key-point recognition using random ferns. IEEE Transactions on Pattern Analysis and Machine Intelligence, 32(3), 448-461.

Digital Library

[19]

Saragih, J., & Goecke, R. (2007). A nonlinear discriminative approach to aam fitting. In International Conference on Computer Vision (ICCV).

[20]

Sauer, P., & Cootes, C. T. T. (2011). Accurate regression procedures for active appearance models. In British Machine Vision Conference (BMVC).

[21]

Shotton, J., Fitzgibbon, A., Cook, M., Sharp, T., Finocchio, M., Moore, R., et al. (2011). Real-time human pose recognition in parts from single depth images. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[22]

Sun, Y., Wang, X., & Tang, X. (2013). Deep convolutional network cascade for facial point detection. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[23]

Tropp, J., & Gilbert, A. (2007). Signal recovery from random measurements via orthogonal matching pursuit. IEEE Transactions on Information Theory, 53(12), 4655-4666.

Digital Library

[24]

Valstar, M., Martinez, B., Binefa, X., & Pantic, M. (2010). Facial point detection using boosted regression and graph models. In IEEE Conference on Computeer Vision and Pattern Recognition (CVPR).

[25]

Vukadinovic, D., & Pantic, M. (2005). Fully automatic facial feature point detection using gabor feature based boosted classifiers. International Conference on Systems, Man and Cybernetics, 2, 1692-1698.

[26]

Xiong, X., De la Torre, F. (2013) Supervised descent method and its applications to face alignment. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR).

[27]

Zhou, S. K., & Comaniciu, D. (2007). Shape regression machine. In Information Processing in Medical Imaging, (pp. 13-25). Heidelberg: Springer.

Cited By

Huang BChen XHuang GLi QLiang GZhong QLi YKe MChen HXie D(2024)Research of Facial Landmark Detection Algorithm based on Deep LearningProceedings of the 5th International Conference on Computer Information and Big Data Applications10.1145/3671151.3671251(561-569)Online publication date: 26-Apr-2024
https://dl.acm.org/doi/10.1145/3671151.3671251
Li YTan GGou C(2024)Cascaded Iterative Transformer for Jointly Predicting Facial Landmark, Occlusion Probability and Head PoseInternational Journal of Computer Vision10.1007/s11263-023-01935-2132:4(1242-1257)Online publication date: 1-Apr-2024
https://dl.acm.org/doi/10.1007/s11263-023-01935-2
Yang ZWang YMa LLi X(2024)Learning Collaborative Reinforcement Attention for 3D Face Reconstruction and Dense AlignmentMultiMedia Modeling10.1007/978-3-031-53311-2_14(184-197)Online publication date: 29-Jan-2024
https://dl.acm.org/doi/10.1007/978-3-031-53311-2_14
Show More Cited By

Index Terms

Face Alignment by Explicit Shape Regression
1. Computing methodologies
  1. Artificial intelligence
    1. Computer vision
      1. Computer vision problems
        Object recognition
  2. Machine learning
    1. Machine learning algorithms
      1. Feature selection

Index terms have been assigned to the content through auto-classification.

Recommendations

Face alignment by Explicit Shape Regression
CVPR '12: Proceedings of the 2012 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)

We present a very efficient, highly accurate, “Explicit Shape Regression” approach for face alignment. Unlike previous regression-based approaches, we directly learn a vectorial regression function to infer the whole facial shape (a set of facial ...
Shape Augmented Regression Method for Face Alignment
ICCVW '15: Proceedings of the 2015 IEEE International Conference on Computer Vision Workshop (ICCVW)

There have been tremendous improvements of the face alignment algorithms, among which the regression framework becomes the most popular one recently. The regression based works start from an initial face shape, and they learn regression models to ...
Joint Local Regressors Learning for Face Alignment

Cascaded regression approaches have been widely applied to computer vision tasks recently, and achieve state-of-the-art performance. In this paper, we consider the problem of face alignment model fitting using cascaded regression and propose a new face ...

Comments

Information & Contributors

Information

Published In

cover image International Journal of Computer Vision

International Journal of Computer Vision Volume 107, Issue 2

April 2014

119 pages

ISSN:0920-5691

Issue’s Table of Contents

Copyright © Copyright © 2014 Springer Science+Business Media New York.

Publisher

Kluwer Academic Publishers

United States

Publication History

Published: 01 April 2014

Author Tags

Qualifiers

Article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

157
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 27 Jul 2024

Other Metrics

View Author Metrics

Citations

Cited By

Huang BChen XHuang GLi QLiang GZhong QLi YKe MChen HXie D(2024)Research of Facial Landmark Detection Algorithm based on Deep LearningProceedings of the 5th International Conference on Computer Information and Big Data Applications10.1145/3671151.3671251(561-569)Online publication date: 26-Apr-2024
https://dl.acm.org/doi/10.1145/3671151.3671251
Li YTan GGou C(2024)Cascaded Iterative Transformer for Jointly Predicting Facial Landmark, Occlusion Probability and Head PoseInternational Journal of Computer Vision10.1007/s11263-023-01935-2132:4(1242-1257)Online publication date: 1-Apr-2024
https://dl.acm.org/doi/10.1007/s11263-023-01935-2
Yang ZWang YMa LLi X(2024)Learning Collaborative Reinforcement Attention for 3D Face Reconstruction and Dense AlignmentMultiMedia Modeling10.1007/978-3-031-53311-2_14(184-197)Online publication date: 29-Jan-2024
https://dl.acm.org/doi/10.1007/978-3-031-53311-2_14
Ding HLiu WYe M(2023)A Data-dependent Approach for High-dimensional (Robust) Wasserstein AlignmentACM Journal of Experimental Algorithmics10.1145/360491028(1-32)Online publication date: 11-Aug-2023
https://dl.acm.org/doi/10.1145/3604910
Bao XCheng ZHe JXiang WLi CSun JLiu HLiu WLuo BGeng YXie XEl Saddik AMei TCucchiara RBertini MTobon Vallejo DAtrey PHossain M(2023)KeyPosS: Plug-and-Play Facial Landmark Detection through GPS-Inspired True-Range MultilaterationProceedings of the 31st ACM International Conference on Multimedia10.1145/3581783.3612366(5746-5755)Online publication date: 26-Oct-2023
https://dl.acm.org/doi/10.1145/3581783.3612366
Zhu CLi XLi JDai STong W(2023)Multi-Sourced Knowledge Integration for Robust Self-Supervised Facial Landmark TrackingIEEE Transactions on Multimedia10.1109/TMM.2022.321226525(6616-6628)Online publication date: 1-Jan-2023
https://dl.acm.org/doi/10.1109/TMM.2022.3212265
Wan JLiu JZhou JLai ZShen LSun HXiong PMin W(2023)Precise Facial Landmark Detection by Reference Heatmap TransformerIEEE Transactions on Image Processing10.1109/TIP.2023.326174932(1966-1977)Online publication date: 1-Jan-2023
https://dl.acm.org/doi/10.1109/TIP.2023.3261749
Thai CNham NTran VBui MNinh HTran H(2023)Multiple Teacher Knowledge Distillation for Head Pose Estimation Without KeypointsSN Computer Science10.1007/s42979-023-02233-x4:6Online publication date: 29-Sep-2023
https://dl.acm.org/doi/10.1007/s42979-023-02233-x
Fan JLiang JLiu HHuan ZHou Z(2023)Robust face alignment via adaptive attention-based graph convolutional networkNeural Computing and Applications10.1007/s00521-023-08531-y35:20(15129-15142)Online publication date: 6-Apr-2023
https://dl.acm.org/doi/10.1007/s00521-023-08531-y
Wu KZhang XZheng XWu SCao YZhou ZMa K(2023)CLN: Complementary Learning Network for 3D Face Reconstruction and AlignmentArtificial Neural Networks and Machine Learning – ICANN 202310.1007/978-3-031-44210-0_13(153-166)Online publication date: 26-Sep-2023
https://dl.acm.org/doi/10.1007/978-3-031-44210-0_13
Show More Cited By

View Options

View options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Issue’s Table of Contents