article

The Good, the Bad, and the Ugly Face Challenge Problem

Authors:

P. Jonathon Phillips,

J. Ross Beveridge,

Bruce A. Draper,

Alice J. O'Toole,

Hassan Sahibzada,

Samuel WeimerAuthors Info & Claims

Image and Vision Computing, Volume 30, Issue 3

Pages 177 - 185

https://doi.org/10.1016/j.imavis.2012.01.004

Published: 01 March 2012 Publication History

Abstract

The Good, the Bad, and the Ugly Face Challenge Problem was created to encourage the development of algorithms that are robust to recognition across changes that occur in still frontal faces. The Good, the Bad, and the Ugly consists of three partitions. The Good partition contains pairs of images that are considered easy to recognize. The base verification rate (VR) is 0.98 at a false accept rate (FAR) of 0.001. The Bad partition contains pairs of images of average difficulty to recognize. For the Bad partition, the VR is 0.80 at a FAR of 0.001. The Ugly partition contains pairs of images considered difficult to recognize, with a VR of 0.15 at a FAR of 0.001. The base performance is from fusing the output of three of the top performers in the FRVT 2006. The design of the Good, the Bad, and the Ugly controls for posevariation, subject aging, and subject ''recognizability.'' Subject recognizability is controlled by having the same number of images of each subject in every partition. This implies that the differences in performance among the partitions are a result of how a face is presented in each image.

References

[1]

Phillips, P.J., Wechsler, H., Huang, J. and Rauss, P., The FERET database and evaluation procedure for face-recognition algorithms. Image Vision Comput. J. v16. 295-306.

[2]

The FERET evaluation methodology for face-recognition algorithms. IEEE Trans. Pattern Anal. Mach. Intell. v22. 1090-1104.

[3]

Phillips, P.J., Flynn, P.J., Scruggs, T., Bowyer, K.W., Chang, J., Hoffman, K., Marques, J., Min, J. and Worek, W., Overview of the face recognition grand challenge. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp. 947-954.

Digital Library

[4]

Phillips, P.J., Scruggs, W.T., O'Toole, A.J., Flynn, P.J., Bowyer, K.W., Schott, C.L. and Sharpe, M., FRVT 2006 and ICE 2006 large-scale results. IEEE Trans. Pattern Anal. Mach. Intell. v32. 831-846.

[5]

Phillips, P.J., Flynn, P.J., Beveridge, J.R., Scruggs, W.T., O'Toole, A.J., Bolme, D., Bowyer, K.W., Draper, B.A., Givens, G.H., Lui, Y.M., Sahibzada, H., Scallan III, J.A. and Weimer, S., Overview of the Multiple Biometrics Grand Challenge. In: Proceedings Third IAPR International Conference on Biometrics,

[6]

Grother, P.J., Quinn, G.W. and Phillips, P.J., MBE 2010: Report on the evaluation of 2D still-image face recognition algorithms. In: NISTIR 7709, National Institute of Standards and Technology,

[7]

Turk, M. and Pentland, A., Eigenfaces for recognition. J. Cogn. Neurosci. v3. 71-86.

[8]

Kirby, M. and Sirovich, L., Application of the Karhunen-Loeve procedure for the characterization of human faces. IEEE Trans. Pattern Anal. Mach. Intell. v12. 103-108.

Digital Library

[9]

Wang, H., Li, S., Wang, Y. and Zhang, J., Self quotient image for face recognition. In: Proceedings, International Conference on Image Processing, volume 2. pp. 1397-1400.

[10]

Lui, Y.M., Bolme, D., Draper, B.A., Beveridge, J.R., Givens, G. and Phillips, P.J., A meta-analysis of face recognition covariates. In: IEEE 3rd International Conference on Biometrics: Theory, Applications, and Systems, Washington, DC,

[11]

Zou, X., Kittler, J. and Messer, K., Illumination invariant face recognition: A survey. In: IEEE International Conference on Biometrics: Theory, Applications, and Systems,

[12]

Basri, R. and Jacobs, D., Lambertian reflectance and linear subspaces. IEEE Trans. Pattern Anal. Mach. Intell. v25. 218-233.

Digital Library

[13]

Sim, T. and Kanade, T., Combining models and exemplars for face recognition: an illuminating example. In: CVPR Workshop on Models versus Exemplars in Computer Vision,

[14]

Beveridge, J.R., Bolme, D.S., Draper, B.A., Givens, G.H., Lui, Y.M. and Phillips, P.J., Quantifying how lighting and focus affect face recognition performance. In: Proceedings IEEE Computer Society and IEEE Biometrics Council Workshop on Biometrics,

[15]

Doddington, G., Ligget, W., Martin, A., Przybocki, M. and Reynolds, D., Sheeps, goats, lambs, and wolves: a statistical analysis of speaker performance in the NIST 1998 recognition evaluation. In: Proceedings ICSLP '98,

[16]

Yager, N. and Dunstone, T., The biometric menagerie. IEEE Trans. Pattern Anal. Mach. Intell. v32. 220-230.

[17]

Teli, M.N., Beveridge, J.R., Phillips, P.J., Givens, G.H., Bolme, D.S. and Draper, B.A., Biometric zoos: theory and experimental evidence. In: International Joint Conference on Biometrics,

[18]

Revisiting Doddington's zoo: a systematic method to assess user-dependent variabilities. In: Proc. of Second Workshop on Multimodal User Authentication (MMUA),

[19]

Ross, A., Rattani, A. and Tistarelli, M., Exploiting the "Doddington zoo" effect in biometric fusion. In: IEEE 3rd International Conference on Biometrics: Theory, Applications, and Systems,

[20]

O'Toole, A.J., Phillips, P.J., An, X. and Dunlop, J., Demographic effects on estimates of automatic face recognition performance. In: Proceedings, Ninth International Conference on Automatic Face and Gesture Recognition,

[21]

A.J. O'Toole, P.J. Phillips, X. An, J. Dunlop, Demographic effects on estimates of automatic face recognition performance, Image and Vision Computing Journal (this issue).

[22]

Beveridge, J.R., Phillips, P.J., Givens, G.H., Draper, B.A., Teli, M.N. and Bolme, D.S., When high-quality face images match poorly. In: Proceedings, Ninth International Conference on Automatic Face and Gesture Recognition,

Cited By

Sero DGarachon IHermens ELiere RBatenburg K(2021)The Study of Three-Dimensional Fingerprint Recognition in Cultural HeritageJournal on Computing and Cultural Heritage 10.1145/346134114:4(1-20)Online publication date: 16-Jul-2021
https://dl.acm.org/doi/10.1145/3461341
Mashhadi ABernhaupt RMueller FVerweij DAndres JMcGrenere JCockburn AAvellino IGoguey ABjørn PZhao SSamson BKocielnik R(2020)A Privacy-Preserving Framework for Collecting Demographic InformationExtended Abstracts of the 2020 CHI Conference on Human Factors in Computing Systems10.1145/3334480.3382800(1-8)Online publication date: 25-Apr-2020
https://dl.acm.org/doi/10.1145/3334480.3382800
Luo ZHu JDeng WShen H(2018)Deep Unsupervised Domain Adaptation for Face Recognition2018 13th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2018)10.1109/FG.2018.00073(453-457)Online publication date: 15-May-2018
https://dl.acm.org/doi/10.1109/FG.2018.00073
Show More Cited By

The Good, the Bad, and the Ugly Face Challenge Problem
1. Computing methodologies
  1. Artificial intelligence
    1. Computer vision
      1. Computer vision problems
      2. Computer vision tasks

Recommendations

Age-Invariant Face Recognition

One of the challenges in automatic face recognition is to achieve temporal invariance. In other words, the goal is to come up with a representation and matching scheme that is robust to changes due to facial aging. Facial aging is a complex process that ...
Face recognition under varying illumination using gradientfaces

In this correspondence, we propose a novel method to extract illumination insensitive features for face recognition under varying lighting called the Gradientfaces. Theoretical analysis shows Gradientfaces is an illumination insensitive measure, and ...
Automatic face analysis system based on face recognition and facial physiognomy
ICHIT'06: Proceedings of the 1st international conference on Advances in hybrid information technology

An automatic face analysis system is proposed which uses face recognition and facial physiognomy. It first detects human's face, extracts its features, and classifies the shape of facial features. It will analyze the person's facial physiognomy and then ...

Comments

Information & Contributors

Information

Published In

cover image Image and Vision Computing

Image and Vision Computing Volume 30, Issue 3

March, 2012

127 pages

ISSN:0262-8856

Issue’s Table of Contents

Copyright © © 2012.

Publisher

Butterworth-Heinemann

United States

Publication History

Published: 01 March 2012

Author Tags

Qualifiers

Article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

9
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 11 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Sero DGarachon IHermens ELiere RBatenburg K(2021)The Study of Three-Dimensional Fingerprint Recognition in Cultural HeritageJournal on Computing and Cultural Heritage 10.1145/346134114:4(1-20)Online publication date: 16-Jul-2021
https://dl.acm.org/doi/10.1145/3461341
Mashhadi ABernhaupt RMueller FVerweij DAndres JMcGrenere JCockburn AAvellino IGoguey ABjørn PZhao SSamson BKocielnik R(2020)A Privacy-Preserving Framework for Collecting Demographic InformationExtended Abstracts of the 2020 CHI Conference on Human Factors in Computing Systems10.1145/3334480.3382800(1-8)Online publication date: 25-Apr-2020
https://dl.acm.org/doi/10.1145/3334480.3382800
Luo ZHu JDeng WShen H(2018)Deep Unsupervised Domain Adaptation for Face Recognition2018 13th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2018)10.1109/FG.2018.00073(453-457)Online publication date: 15-May-2018
https://dl.acm.org/doi/10.1109/FG.2018.00073
Patterson ESimpson D(2017)Dataset and Metrics for Adult Age-Progression EvaluationProceedings of the 2017 ACM Southeast Conference10.1145/3077286.3077304(248-251)Online publication date: 13-Apr-2017
https://dl.acm.org/doi/10.1145/3077286.3077304
Alsalibi BVenkat IAl-Betar M(2017)A membrane-inspired bat algorithm to recognize faces in unconstrained scenariosEngineering Applications of Artificial Intelligence10.1016/j.engappai.2017.06.01864:C(242-260)Online publication date: 1-Sep-2017
https://dl.acm.org/doi/10.1016/j.engappai.2017.06.018
Tang KXiao XLi Z(2015)Exploit more information of the sample for representation based face recognitionInternational Journal of Wireless and Mobile Computing10.1504/IJWMC.2015.0709388:4(401-405)Online publication date: 1-Aug-2015
https://dl.acm.org/doi/10.1504/IJWMC.2015.070938
Hassaballah MAly S(2015)Face recognitionIET Computer Vision10.1049/iet-cvi.2014.00849:4(614-626)Online publication date: 1-Aug-2015
https://dl.acm.org/doi/10.1049/iet-cvi.2014.0084
Givens GBeveridge JLui YBolme DDraper BPhillips P(2013)Biometric face recognitionWIREs Computational Statistics10.1002/wics.12625:4(288-308)Online publication date: 1-Jul-2013
https://dl.acm.org/doi/10.1002/wics.1262
O'Toole AAn XDunlop JNatu VPhillips P(2012)Comparing face recognition algorithms to humans on challenging tasksACM Transactions on Applied Perception10.1145/2355598.23555999:4(1-13)Online publication date: 22-Oct-2012
https://dl.acm.org/doi/10.1145/2355598.2355599

View Options

View options

Figures

Tables

Media

View Issue’s Table of Contents