Abstract
The aim in this paper is to use principal geodesic analysis to model the statistical variations for sets of facial needle maps. We commence by showing how to represent the distribution of surface normals using the exponential map. Shape deformations are described using principal geodesic analysis on the exponential map. Using ideas from robust statistics we show how this deformable model may be fitted to facial images in which there is significant self-shadowing. Moreover, we demonstrate that the resulting shape-from-shading algorithm can be used to recover accurate facial shape and albedo from real world images. In particular, the algorithm can effectively fill-in the facial surface when more than 30% of its area is subject to self-shadowing. To investigate the utility of the shape parameters delivered by the method, we conduct experiments with illumination insensitive face recognition. We present a novel recognition strategy in which similarity is measured in the space of the principal geodesic parameters. We also use the recovered shape information to generate illumination normalized prototype images on which recognition can be performed. Finally we show that, from a single input image, we are able to generate the basis images employed by a number of well known illumination-insensitive recognition algorithms. We also demonstrate that the principal geodesics provide an efficient parameterization of the space of harmonic basis images.
Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.References
USF HumanID. 3D Face Database, Courtesy of Sudeep. Sarkar, University of South Florida, Tampa, FL.
Atick, J. J., Griffin, P. A., & Redlich, A. N. (1996). Statistical approach to SFS: Reconstruction of 3D face surfaces from single 2D images. Neural Computation, 8(6), 1321–1340.
Basri, R., & Jacobs, D. W. (2003). Lambertian reflectance and linear subspaces. IEEE Transactions on Pattern Analysis and Machine Intelligence, 25(2), 218–233.
Belhumeur, P.N., Kriegman, D.J., & Yuille, A.L. (1999). The bas-relief ambiguity. International Journal of Computer Vision, 35(1), 33–44.
Blanz, V., Basso, C., Poggio, T., & Vetter, T. (2003). Reanimating faces in images and video. In Proceedings of EUROGRAPHICS (pp. 641–650).
Blanz, V., & Vetter, T. (1999). A morphable model for the synthesis of 3D faces. In Proceedings of SIGGRAPH (pp. 187–194).
Blanz, V., & Vetter, T. (2003). Face recognition based on fitting a 3D morphable model. IEEE Transactions on Pattern Analysis and Machine Intelligence, 25(9), 1063–1074.
Bronstein, A., Bronstein, M., & Kimmel, R. (2005). Three-dimensional face recognition. International Journal of Computer Vision, 64(1), 5–30.
Castelán, M., & Hancock, E. R. (2006). Acquiring height data from a single image of a face using local shape indicators. Computer Vision and Image Understanding, 103(1), 64–79.
Dovgard, R., & Basri, R. (2004). Statistical symmetric shape from shading for 3D structure recovery of faces. In Proceedings of ECCV (Vol. 2, pp. 99–113).
Dupuis, P., & Oliensis, J. (1994). An optimal control formulation and related numerical methods for a problem in shape reconstruction. Annals of Applied Probability, 4(2), 287–346.
Finlayson, G. D., Drew, M. S., & Lu, C. (2004). Intrinsic images by entropy minimization. In Proceedings of ECCV (pp. 582–595).
Fisher, N. I. (1985). Spherical medians. Journal of the Royal Statistical Society, Series B, 47(2), 342–348.
Fletcher, P. T., Joshi, S., Lu, C., & Pizer, S. M. (2004). Principal geodesic analysis for the study of nonlinear statistics of shape. IEEE Transactions on Medical Imaging, 23(8), 995–1005.
Frankot, R. T., & Chellappa, R. (1988). A method for enforcing integrability in shape from shading algorithms. IEEE Transactions on Pattern Analysis and Machine Intelligence, 10(4), 439–451.
Fua, P., & Leclerc, Y. G. (1994). Using 3-dimensional meshes to combine image-based and geometry-based constraints. In Proceedings of ECCV (pp. 281–291).
Georghiades, A. (2003). Recovering 3-d shape and reflectance from a small number of photographs. In Eurographics symposium on rendering (pp. 230–240).
Georghiades, A. S., Belhumeur, P. N., & Kriegman, D. J. (2001). From few to many: Illumination cone models for face recognition under variable lighting and pose. IEEE Transactions on Pattern Analysis and Machine Intelligence, 23(6), 643–660.
Giles, P. T. (2001). Remote sensing and cast shadows in mountainous terrain. Photogrammetric Engineering & Remote Sensing, 67(7), 833–840.
Gregory, R. L. (1997). Knowledge in perception and illusion. Philosophical Transactions of the Royal Society of London, Series B, 352, 1121–1128.
Hill, H., & Bruce, V. (1996). Effects of lighting on the perception of facial surfaces. Journal of Experimental Psychology: Human Perception and Performance, 22(4), 986–1004.
Huber, P. (1981). Robust statistics. Chichester: Wiley.
Lee, K., Ho, J., & Kriegman, D. (2005). Acquiring linear subspaces for face recognition under variable lighting. IEEE Transactions on Pattern Analysis and Machine Intelligence, 27(5), 1–15.
Lenglet, C., Rousson, M., Deriche, R., Faugeras, O., Lehericy, S., & Ugurbil, K. (2005). A Riemannian approach to diffusion tensor images segmentation. In Proceedings of the information processing in medical imaging (pp. 591–602).
Levine, M. D., & Bhattacharyya, J. (2005). Removing shadows. Pattern Recognition Letters, 26(3), 251–265.
Mardia, K. V., & Jupp, P. E. (2000). Directional statistics. New York: Wiley.
Marr, D. (1982). Vision. San Francisco: Freeman.
Nishino, K., Belhumeur, P. N., & Nayar, S. K. (2005). Using eye reflections for face recognition under varying illumination. In Proceedings of ICCV (Vol. 1, pp. 519–526).
Pennec, X. (1999). Probabilities and statistics on Riemannian manifolds: basic tools for geometric measurements. In Proceedings of the IEEE workshop on nonlinear signal and image processing.
Pennec, X. (2004). Probabilities and statistics on Riemannian manifolds: a geometric approach (Technical Report RR-5093). INRIA.
Prados, E., & Faugeras, O. (2004). A rigorous and realistic shape from shading method and some of its applications (Technical Report RR-5133). INRIA.
Prados, E., & Faugeras, O. D. (2004). Unifying approaches and removing unrealistic assumptions in shape from shading: mathematics can help. In Proceedings of ECCV (pp. 141–154).
Samaras, D., & Metaxas, D. (2003). Illumination constraints in deformable models for shape and light direction estimation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 25(2), 247–264.
Smith, W. A. P., & Hancock, E. R. (2006). Recovering facial shape using a statistical model of surface normal direction. IEEE Transactions on Pattern Analysis and Machine Intelligence, 28(12), 1914–1930.
Turk, M., & Pentland, A. (1991). Face recognition using eigenfaces. In Proceedings of CVPR (pp. 586–591).
Worthington, P. L., & Hancock, E. R. (1999). New constraints on data-closeness and needle map consistency for shape-from-shading. IEEE Transactions on Pattern Analysis and Machine Intelligence, 21(12), 1250–1267.
Zhang, L., Wang, S., & Samaras, D. (2005). Face synthesis and recognition under arbitrary unknown lighting using a spherical harmonic basis morphable model. In Proceedings of CVPR (pp. 209–216).
Zhao, W. Y., & Chellappa, R. (2000). Illumination-insensitive face recognition using symmetric SFS. In Proceedings of CVPR (pp. 286–293).
Zhao, W. Y., & Chellappa, R. (2001). Symmetric shape-from-shading using self-ratio image. International Journal of Computer Vision, 45, 55–75.
Zhou, S., & Chellappa, R. (2003). Rank constrained recognition under unknown illuminations. In Proceedings of the IEEE international workshop on analysis and modeling of faces and gestures (pp. 11–18).
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Smith, W.A.P., Hancock, E.R. Facial Shape-from-shading and Recognition Using Principal Geodesic Analysis and Robust Statistics. Int J Comput Vis 76, 71–91 (2008). https://doi.org/10.1007/s11263-007-0074-8
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11263-007-0074-8