Abstract
We present a statistical model for 3D human faces in varying expression, which decomposes the surface of the face using a wavelet transform, and learns many localized, decorrelated multilinear models on the resulting coefficients. Using this model we are able to reconstruct faces from noisy and occluded 3D face scans, and facial motion sequences. Accurate reconstruction of face shape is important for applications such as tele-presence and gaming. The localized and multi-scale nature of our model allows for recovery of fine-scale detail while retaining robustness to severe noise and occlusion, and is computationally efficient and scalable. We validate these properties experimentally on challenging data in the form of static scans and motion sequences. We show that in comparison to a global multilinear model, our model better preserves fine detail and is computationally faster, while in comparison to a localized PCA model, our model better handles variation in expression, is faster, and allows us to fix identity parameters for a given subject.
This work has partially been funded by the German Research Foundation (DFG).
Chapter PDF
Similar content being viewed by others
References
Amberg, B., Knothe, R., Vetter, T.: Expression invariant 3D face recognition with a morphable model. In: FG, pp. 1–6 (2008)
Arya, A., Mount, D.: Approximate nearest neighbor queries in fixed dimensions. In: SODA, pp. 271–280 (1993), http://www.cs.umd.edu/~mount/ANN/
Bertram, M., Duchaineau, M., Hamann, B., Joy, K.I.: Generalized B-Spline subdivision-surface wavelets for geometry compression. TVCG 10(3), 326–338 (2004)
Blanz, V., Scherbaum, K., Seidel, H.P.: Fitting a morphable model to 3D scans of faces. In: ICCV, pp. 1–8 (2007)
Blanz, V., Vetter, T.: A morphable model for the synthesis of 3D faces. In: SIGGRAPH, pp. 187–194 (1999)
Bolkart, T., Brunton, A., Salazar, A., Wuhrer, S.: Statistical 3D shape models of human faces (2013), http://statistical-face-models.mmci.uni-saarland.de/
Bolkart, T., Wuhrer, S.: Statistical analysis of 3D faces in motion. In: 3DV, pp. 103–110 (2013)
Brunton, A., Shu, C., Lang, J., Dubois, E.: Wavelet model-based stereo for fast, robust face reconstruction. In: CRV, pp. 347–354 (2011)
Chen, Y., Liu, Z., Zhang, Z.: Tensor-based human body modeling. In: CVPR, pp. 105–112 (2013)
Creusot, C., Pears, N., Austin, J.: A machine-learning approach to keypoint detection and landmarking on 3D meshes. IJCV 102(1-3), 146–179 (2013)
Dale, K., Sunkavalli, K., Johnson, M.K., Vlasic, D., Matusik, W., Pfister, H.: Video face replacement. TOG 30(6), 130:1–130:10 (2011)
Davatzikos, C., Tao, X., Shen, D.: Hierarchical active shape models, using the wavelet transform. TMI 22(3), 414–423 (2003)
ter Haar, F.B., Veltkamp, R.C.: 3D face model fitting for recognition. In: Forsyth, D., Torr, P., Zisserman, A. (eds.) ECCV 2008, Part IV. LNCS, vol. 5305, pp. 652–664. Springer, Heidelberg (2008)
Kakadiaris, I., Passalis, G., Toderici, G., Murtuza, M., Lu, Y., Karamelpatzis, N., Theoharis, T.: Three-dimensional face recognition in the presence of facial expressions: An annotated deformable model approach. TPAMI 29(4), 640–649 (2007)
Kobbelt, L., Campagna, S., Vorsatz, J., Seidel, H.P.: Interactive multi-resolution modeling on arbitrary meshes. In: SIGGRAPH 1998, pp. 105–114 (1998)
Lathauwer, L.D.: Signal processing based on multilinear algebra. Ph.D. thesis, K.U. Leuven, Belgium (1997)
Lecron, F., Boisvert, J., Mahmoudi, S., Labelle, H., Benjelloun, M.: Fast 3D spine reconstruction of postoperative patients using a multilevel statistical model. In: Ayache, N., Delingette, H., Golland, P., Mori, K. (eds.) MICCAI 2012, Part II. LNCS, vol. 7511, pp. 446–453. Springer, Heidelberg (2012)
Li, Y., Tan, T.S., Volkau, I., Nowinski, W.: Model-guided segmentation of 3D neuroradiological image using statistical surface wavelet model. In: CVPR, pp. 1–7 (2007)
Liu, D., Nocedal, J.: On the limited memory method for large scale optimization. Math. Prog.: Ser. A, B 45(3), 503–528 (1989)
Mpiperis, I., Malassiotis, S., Strintzis, M.G.: Bilinear models for 3-D face and facial expression recognition. TIFS 3, 498–511 (2008)
Nain, D., Haker, S., Bobick, A., Tannenbaum, A.R.: Multiscale 3D shape analysis using spherical wavelets. In: Duncan, J.S., Gerig, G. (eds.) MICCAI 2005. LNCS, vol. 3750, pp. 459–467. Springer, Heidelberg (2005)
Patel, A., Smith, W.: 3D morphable face models revisited. In: CVPR, pp. 1327–1334 (2009)
Salazar, A., Wuhrer, S., Shu, C., Prieto, F.: Fully automatic expression-invariant face correspondence. MVAP 25(4), 859–879 (2014)
Savran, A., Alyüz, N., Dibeklioğlu, H., Çeliktutan, O., Gökberk, B., Sankur, B., Akarun, L.: Bosphorus database for 3D face analysis. In: Schouten, B., Juul, N.C., Drygajlo, A., Tistarelli, M. (eds.) BIOID 2008. LNCS, vol. 5372, pp. 47–56. Springer, Heidelberg (2008)
Schröder, P., Sweldens, W.: Spherical wavelets: Efficiently representing functions on the sphere. In: SIGGRAPH, pp. 161–172 (1995)
De Smet, M., Van Gool, L.: Optimal regions for linear model-based 3D face reconstruction. In: Kimmel, R., Klette, R., Sugimoto, A. (eds.) ACCV 2010, Part III. LNCS, vol. 6494, pp. 276–289. Springer, Heidelberg (2011)
Sweldens, W.: The lifting scheme: A custom-design construction of biorthogonal wavelets. Appl. Comp. Harm. Anal. 3(2), 186–200 (1996)
Tam, G., Cheng, Z.Q., Lai, Y.K., Langbein, F., Liu, Y., Marshall, D., Martin, R., Sun, X.F., Rosin, P.: Registration of 3D point clouds and meshes: A survey from rigid to nonrigid. TVCG 19(7), 1199–1217 (2013)
Vasilescu, M.A.O., Terzopoulos, D.: Multilinear analysis of image ensembles: TensorFaces. In: Heyden, A., Sparr, G., Nielsen, M., Johansen, P. (eds.) ECCV 2002, Part I. LNCS, vol. 2350, pp. 447–460. Springer, Heidelberg (2002)
Vlasic, D., Brand, M., Pfister, H., Popović, J.: Face transfer with multilinear models. TOG 24(3), 426–433 (2005)
Yang, F., Bourdev, L., Wang, J., Shechtman, E., Metaxas, D.: Facial expression editing in video using a temporally-smooth factorization. In: CVPR, pp. 861–868 (2012)
Yin, L., Chen, X., Sun, Y., Worm, T., Reale, M.: A high-resolution 3D dynamic facial expression database. In: FG, pp. 1–6 (2008)
Yin, L., Wei, X., Sun, Y., Wang, J., Rosato, M.J.: A 3D facial expression database for facial behavior research. In: FG, pp. 211–216 (2006)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
1 Electronic Supplementary Material
Rights and permissions
Copyright information
© 2014 Springer International Publishing Switzerland
About this paper
Cite this paper
Brunton, A., Bolkart, T., Wuhrer, S. (2014). Multilinear Wavelets: A Statistical Shape Space for Human Faces. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds) Computer Vision – ECCV 2014. ECCV 2014. Lecture Notes in Computer Science, vol 8689. Springer, Cham. https://doi.org/10.1007/978-3-319-10590-1_20
Download citation
DOI: https://doi.org/10.1007/978-3-319-10590-1_20
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-10589-5
Online ISBN: 978-3-319-10590-1
eBook Packages: Computer ScienceComputer Science (R0)