research-article

Structure-aware transfer of facial blendshapes

Authors:

Christos Mousas,

Christos-Nikolaos AnagnostopoulosAuthors Info & Claims

SCCG '15: Proceedings of the 31st Spring Conference on Computer Graphics

Pages 55 - 62

https://doi.org/10.1145/2788539.2788546

Published: 22 April 2015 Publication History

Abstract

This paper presents a novel mesh deformation method for transferring reference facial blendshapes to a source face model. The presented method uses information provided by a source and a reference face model to analyze the mesh similarities and, consequently, to transfer the deformation. This is achieved by considering the distribution of vertices between the two face models in conjunction with the gradients of the meshes. Both of these components are assigned to an optimization function that is solved in two steps: (i) distribution transfer and (ii) gradient maintenance optimization. Finally, in order to analyze the efficiency of such a method, the presented method was evaluated against previous solutions in terms of deformation transfer error, and computational time.

References

[1]

Agarwala, A. 2007. Efficient gradient-domain compositing using quadtrees. ACM Transactions on Graphics 26, 3 (August), Article No. 94.

Digital Library

[2]

Baran, I., and Popović, J. 2007. Automatic rigging and animation of 3d characters. ACM Transactions on Graphics 26, 3 (August), Article No. 72.

Digital Library

[3]

Bergeron, P., and Lachapelle, P. 1985. Controlling facial expressions and body movements in the computer generated animated short "tony de peltrie". In SIGGRAPH Tutorial Notes, Advanced Computer Animation Course.

[4]

Bolz, J., Farmer, I., Grinspun, E., and Schröoder, P. 2003. Sparse matrix solvers on the gpu: conjugate gradients and multigrid. ACM Transactions on Graphics 22, 3 (July), 917--924.

Digital Library

[5]

Bouaziz, S., Wang, Y., and Pauly, M. 2013. Online modeling for realtime facial animation. ACM Transactions on Graphics 32, 3, Article No. 40.

Digital Library

[6]

Bradley, D., Heidrich, W., Popa, T., and Sheffer, A. 2010. High resolution passive facial performance capture. ACM Transactions on Graphics 29, 4, Article No. 41.

Digital Library

[7]

Deng, Z., and Noh, J. 2007. Computer facial animation: A survey. In Data-Driven 3D Facial Animation, Springer London, 1--28.

[8]

Deng, Z., Chiang, P. Y., Fox, P., and Neumann, U. 2006. Animating blendshape faces by cross-mapping motion capture data. In Symposium on Interactive 3D Graphics and Games, 43--48.

Digital Library

[9]

Deutscher, J., and Reid, I. 2005. Articulated body motion capture by stochastic search. International Journal of Computer Vision 61, 2, 185--205.

Digital Library

[10]

Furukawa, Y., and Ponce, J. 2009. Dense 3d motion capture for human faces. In IEEE Conference on Computer Vision and Pattern Recognition, IEEE Press, 1674--1681.

[11]

Goodnight, N., Woolley, C., Lewin, G., Luebke, D., and Humphreys, G. 2003. A multigrid solver for boundary value problems using programmable graphics hardware. In ACM SIGGRAPH/EUROGRAPHICS Conference on Graphics Hardware, Eurographics Association, 102--111.

Digital Library

[12]

Guenter, B., Grimm, C., Wood, D., Malvar, H., and Pighin, F. 1998. Making faces. In Annual Conference on Computer Graphics and Interactive Techniques, ACM, 55--66.

Digital Library

[13]

Huang, J., Shi, X., Liu, X., Zhou, K., Wei, L. Y., Teng, S. H., Bao, H., Guo, B., and Shum, H. Y. 2006. Sub-space gradient domain mesh deformation. ACM Transactions on Graphics 25, 3, 1126--1134.

Digital Library

[14]

Huang, Z., Yao, J., Zhong, Z., Liu, Y., and Guo, X. 2014. Sparse localized decomposition of deformation gradients. Computer Graphics Forum 33, 7 (October), 239--248.

Digital Library

[15]

Jaiman, R. K., Jiao, X., Geubelle, P. H., and Loth, E. 2006. Conservative load transfer along curved fluid--solid interface with non-matching meshes. Journal of Computational Physics 218, 1, 372--397.

Digital Library

[16]

Lewis, J. P., Anjyo, K., Rhee, T., Zhang, M., Pighin, F., and Deng, Z. 2014. Practice and theory of blendshape facial models. In Eurographics - State of the Art Reports, Eurographics Association, 199--218.

[17]

Li, H., Adams, B., Guibas, L. J., and Pauly, M. 2009. Robust single-view geometry and motion reconstruction. ACM Transactions on Graphics 28, 5 (December), Article No. 175.

Digital Library

[18]

Li, H., Weise, T., and Pauly, M. 2010. Example-based facial rigging. ACM Transactions on Graphics 29, 4, Article No. 32.

Digital Library

[19]

Li, H., Yu, J., Ye, Y., and Bregler, C. 2013. Realtime facial animation with on-the-fly correctives. ACM Transactions on Graphics 32, 4, Article No. 42.

Digital Library

[20]

Lin, I. C., and Ouhyoung, M. 2005. Mirror mocap: Automatic and efficient capture of dense 3d facial motion parameters from video. The Visual Computer 21, 6, 355--372.

[21]

Monzani, J. S., Baerlocher, P., Boulic, R., and Thalmann, D. 2000. Using an intermediate skeleton and inverse kinematics for motion retargeting. Computer Graphics Forum 19, 3 (September), 11--19.

[22]

Mpiperis, I., Malassiotis, S., and Strintzis, M. G. 2008. Bilinear elastically deformable models with application to 3d face and facial expression recognition. In IEEE International Conference on Automatic Face and Gesture Recognition, IEEE Press, 1--8.

[23]

Noh, J. Y., and Neumann, U. 2001. Expression cloning. In Annual Conference on Computer Graphics and Interactive Techniques, 277--288.

Digital Library

[24]

Osipa, J. 2010. Stop staring: facial modeling and animation done right (3rd edition). John Wiley and Sons.

Digital Library

[25]

Parke, F. I. 1972. Computer generated animation of faces. Master's thesis, University of Utah.

[26]

Pyun, H., Kim, Y., Chae, W., Kang, H. W., and Shin, S. Y. 2003. An example-based approach for facial expression cloning. In ACM SIGGRAPH/Eurographics Symposium on Computer Animation, Eurographics Association, 167--176.

Digital Library

[27]

Rachev, S. T., and Rüschendorf, L. 1998. Mass Transportation Problems: Volume I: Theory. Springer.

[28]

Saito, J. 2013. Smooth contact-aware facial blendshapes transfer. In Symposium on Digital Production, ACM Press, 7--12.

Digital Library

[29]

Sumner, R. W., and Popović, J. 2004. Deformation transfer for triangle meshes. ACM Transactions on Graphics 23, 3 (August), 399--405.

Digital Library

[30]

Sumner, R. W. 2005. Mesh modification using deformation gradients. Ph.d. dissertation, Massachusetts Institute of Technology.

Digital Library

[31]

Wang, J., Yin, L., Wei, X., and Sun, Y. 2006. 3d facial expression recognition based on primitive surface feature distribution. In IEEE Conference on Computer Vision and Pattern Recognition, IEEE Press, 1399--1406.

Digital Library

[32]

Weise, T., Li, H., Van Gool, L., and Pauly, M. 2009. Face/off: Live facial puppetry. In ACM SIGGRAPH/Eurographics Symposium on Computer Animation, Eurographics Association, 7--16.

Digital Library

[33]

Weise, T., Bouaziz, S., Li, H., and Pauly, M. 2011. Real-time performance-based facial animation. ACM Transactions on Graphics 30, 4, Article No. 77.

Digital Library

[34]

Wenyin, L., and Dori, D. 1998. A survey of non-thinning based vectorization methods. In Advances in Pattern Recognition, Springer Berlin Heidelberg, 230--241.

Digital Library

[35]

Yu, Y., Zhou, K., Xu, D., Shi, X., Bao, H., Guo, B., and Shum, H. Y. 2004. Mesh editing with poisson-based gradient field manipulation. ACM Transactions on Graphics 23, 3 (August), 644--651.

Digital Library

[36]

Zhang, S., and Huang, P. 2004. High-resolution, real-time 3d shape acquisition. In IEEE Conference on Computer Vision and Pattern Recognition, IEEE.

Digital Library

[37]

Zhou, K., Xu, W., Tong, Y., and Desbrun, M. 2010. Deformation transfer to multicomponent objects. Computer Graphics Forum 29, 2 (May), 319--325.

Cited By

Dou YMukai T(2024)Facial Animation Retargeting by Unsupervised Learning of Graph Convolutional Networks2024 Nicograph International (NicoInt)10.1109/NICOInt62634.2024.00022(69-75)Online publication date: 14-Jun-2024
https://doi.org/10.1109/NICOInt62634.2024.00022
Cipiloglu Yildiz Z(2023)Learning a crowd-powered perceptual distance metric for facial blendshapesEURASIP Journal on Image and Video Processing10.1186/s13640-023-00609-w2023:1Online publication date: 15-May-2023
https://doi.org/10.1186/s13640-023-00609-w
Somepalli MCharan MShruthi SPalaniswamy S(2021)Implementation of Single Camera Markerless Facial Motion Capture using Blendshapes2021 IEEE International Conference on Computation System and Information Technology for Sustainable Solutions (CSITSS)10.1109/CSITSS54238.2021.9683460(1-6)Online publication date: 16-Dec-2021
https://doi.org/10.1109/CSITSS54238.2021.9683460
Show More Cited By

Index Terms

Structure-aware transfer of facial blendshapes
1. Computing methodologies
  1. Computer graphics
    1. Animation
    2. Shape modeling
2. Theory of computation
  1. Randomness, geometry and discrete structures

Recommendations

Smooth contact-aware facial blendshapes transfer
DigiPro '13: Proceedings of the Symposium on Digital Production

In this paper, we propose a method to improve the quality of the facial blendshapes synthesized using deformation transfer by introducing the idea of contact and smoothness awareness. Simply applying deformation transfer for face models easily results ...
Customizing blendshapes to capture facial details
Abstract
Blendshape technique is an effective tool in the computer facial animation. Every character requires its own unique blendshapes to cover numerous facial expressions in the Visual Effects industry. Despite outstanding advances in this area, ...
Dynamic facial asset and rig generation from a single scan

The creation of high-fidelity computer-generated (CG) characters for films and games is tied with intensive manual labor, which involves the creation of comprehensive facial assets that are often captured using complex hardware. To simplify and ...

Comments

Information & Contributors

Information

Published In

cover image ACM Other conferences

SCCG '15: Proceedings of the 31st Spring Conference on Computer Graphics

April 2015

152 pages

ISBN:9781450336932

DOI:10.1145/2788539

Conference Chair:
Joaquim Jorge
Universidade de Lisboa, Portugal
,
Program Chairs:
Luís Paulo Santos
Universidade do Minho, Portugal
,
Roman Ďurikovič
Comenius University, Slovakia

Copyright © 2015 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

In-Cooperation

SIGGRAPH: ACM Special Interest Group on Computer Graphics and Interactive Techniques

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 22 April 2015

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

SCCG'15

SCCG'15: Spring Conference on Computer Graphics

April 22 - 24, 2015

Smolenice, Slovakia

Acceptance Rates

Overall Acceptance Rate 67 of 115 submissions, 58%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

7
Total Citations
View Citations
181
Total Downloads

Downloads (Last 12 months)18
Downloads (Last 6 weeks)1

Reflects downloads up to 23 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Dou YMukai T(2024)Facial Animation Retargeting by Unsupervised Learning of Graph Convolutional Networks2024 Nicograph International (NicoInt)10.1109/NICOInt62634.2024.00022(69-75)Online publication date: 14-Jun-2024
https://doi.org/10.1109/NICOInt62634.2024.00022
Cipiloglu Yildiz Z(2023)Learning a crowd-powered perceptual distance metric for facial blendshapesEURASIP Journal on Image and Video Processing10.1186/s13640-023-00609-w2023:1Online publication date: 15-May-2023
https://doi.org/10.1186/s13640-023-00609-w
Somepalli MCharan MShruthi SPalaniswamy S(2021)Implementation of Single Camera Markerless Facial Motion Capture using Blendshapes2021 IEEE International Conference on Computation System and Information Technology for Sustainable Solutions (CSITSS)10.1109/CSITSS54238.2021.9683460(1-6)Online publication date: 16-Dec-2021
https://doi.org/10.1109/CSITSS54238.2021.9683460
Niu ZLi HLi YMei YYang J(2020)An Adaptive Face Image Inpainting Algorithm Based on Feature SymmetrySymmetry10.3390/sym1202019012:2(190)Online publication date: 22-Jan-2020
https://doi.org/10.3390/sym12020190
Alsaggaf WTsaramirsis GAl-Malki NKhan FAlmasry MAbdulhalim Serafi MAlmarzuqi A(2020)Association of Game Events with Facial Animations of Computer-Controlled Virtual Characters Based on Probabilistic Human Reaction ModelingApplied Sciences10.3390/app1016563610:16(5636)Online publication date: 14-Aug-2020
https://doi.org/10.3390/app10165636
Liu KChen JChang K(2019)A Study of Facial Features of American and Japanese Cartoon CharactersSymmetry10.3390/sym1105066411:5(664)Online publication date: 12-May-2019
https://doi.org/10.3390/sym11050664
Nonis FDagnes NMarcolin FVezzetti E(2019)3D Approaches and Challenges in Facial Expression Recognition Algorithms—A Literature ReviewApplied Sciences10.3390/app91839049:18(3904)Online publication date: 18-Sep-2019
https://doi.org/10.3390/app9183904
Ouzounis CKilias AMousas CJaillet FZara F(2017)Kernel projection of latent structures regression for facial animation retargetingProceedings of the 13th Workshop on Virtual Reality Interactions and Physical Simulations10.2312/vriphys.20171084(59-65)Online publication date: 23-Apr-2017
https://dl.acm.org/doi/10.2312/vriphys.20171084

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten