research-article

Content retargeting using parameter-parallel facial layers

Authors:

Natasha Kholgade,

Yaser SheikhAuthors Info & Claims

SCA '11: Proceedings of the 2011 ACM SIGGRAPH/Eurographics Symposium on Computer Animation

Pages 195 - 204

https://doi.org/10.1145/2019406.2019433

Published: 05 August 2011 Publication History

Abstract

Facial motion retargeting approaches often transfer expressions by establishing correspondences between shared units of motion, such as action units, or spatial correspondences of landmarks between the source actor and target character faces. When the actor and character are structurally dissimilar, shared units of motion or spatial landmarks may not exist, and subtle styles of performance may differ. We present a method to deconstruct the content of an actor's facial expression into three parameter-parallel layers using a composition function, transfer the content to equivalent parameter-parallel layers for the character, and reconstruct the character's expression using the same composition function. Our algorithm uses the same parameter-parallel layered model of facial expression for both the actor and character, separating the content of facial expressions into emotion, speech, and eye-blink layers. Facial motion in each layer is embedded in simplicial bases, each of which encodes semantically significant configurations of the face. We show the transfer of facial motion capture and video-based tracking of the eyes and mouth of an actor to a number of faces with dissimilar facial structure and expressive disposition.

Supplementary Material

JPG File (p195-kholgade.jpg)

Download
13.17 KB

MOV File (p195-kholgade.mov)

Download
53.49 MB

References

[1]

{BFJ*00} Buck I., Finkelstein A., Jacobs C., Klein A., Salesin D. H., Seims J., Szeliski R., Toyama K.: Performance-driven hand-drawn animation. First International Symposium on Non Photorealistic Animation and Rendering (2000), 101--108. 3

Digital Library

[2]

{BLB*08} Bickel B., Lang M., Botsch M., Otaduy M. A., Gross M.: Pose-space animation and transfer of facial details. Proceedings of the ACM SIGGRAPH/Eurographics Symposium on Computer Animation (2008). 3

Digital Library

[3]

{BLCD02} Bregler C., Loeb L., Chuang E., Deshpande H.: Turning to the masters: Motion capturing cartoons. Proceedings of the 29th Annual Conference on Computer Graphics and Interactive Techniques (2002). 3

Digital Library

[4]

{BVGP09} Baran I., Vlasic D., Grinspun E., Popovic J.: Semantic deformation transfer. ACM Transactions on Graphics 28, 3 (2009). 3

Digital Library

[5]

{CB02} Chuang E., Bregler C.: Performance driven facial animation using blendshapes interpolation. Technical Report CS-TR-2002-02, Stanford University (2002). 3

[6]

{CB05} Chuang E., Bregler C.: Mood swings: Expressive speech animation. ACM Transactions on Graphics 24, 2 (2005), 331--347. 3

Digital Library

[7]

{CBK*06} Curio C., Breidt M., Kleiner M., Vuong Q. C., Giese M. A., Bulthoff H. H.: Semantic 3d motion retargeting for facial animation. Proceedings of the 3rd symposium on Applied perception in graphics and visualization (2006). 3

Digital Library

[8]

{CFP03} Cao Y., Faloutsos P., Pighin F.: Unsupervised learning for speech motion editing. In Proceedings of the ACM SIGGRAPH/Eurographics symposium on Computer animation (Aire-la-Ville, Switzerland, Switzerland, 2003), Eurographics Association, pp. 225--231. 3

Digital Library

[9]

{CLK01} Choe B., Lee H., Ko H.-S.: Performance-driven muscle-based facial animation. Journal of Visualization and Computer Animation 12, 2 (2001), 67--79. 2, 3

[10]

{DCFN06} Deng Z., Chiang P.-Y., Fox P., Neumann U.: Animating blendshape faces by cross-mapping motion capture data. Proceedings of the Symposium on Interactive 3D graphics and games (2006). 3

Digital Library

[11]

{EFH02} Ekman P., Friesen W. V., Hager J. C.: Facial Action Coding System: The Manual. 2002. 3

[12]

{Gle98} Gleicher M.: Retargetting motion to new characters. Proceedings of ACM SIGGRAPH (1998). 2

Digital Library

[13]

{JTDP03} Joshi P., Tien W. C., Desbrun M., Pighin F.: Learning controls for blend shape based realistic facial animation. Proceedings of the ACM SIGGRAPH/Eurographics Symposium on Computer Animation (2003). 3

Digital Library

[14]

{LWP10} Li H., Weise T., Pauly M.: Example-based facial rigging. ACM Transactions on Graphics 29, 4 (2010). 2

Digital Library

[15]

{MB04} Matthews I., Baker S.: Active appearance models revisited. International Journal of Computer Vision 60, 2 (2004), 134--164. 2, 4, 7

Digital Library

[16]

{MJC*08} Ma W.-C., Jones A., Chiang J.-Y., Hawkins T., Frederiksen S., Peers P., Vukovic M., Ouhyoung M., Debevec P.: Facial performance synthesis using deformation-driven polynomial displacement maps. ACM Transactions on Graphics 27, 5 (2008). 2, 3

Digital Library

[17]

{NJ04} Na K., Jung M.: Hierarchical retargeting of fine facial motions. Eurographics 23, 3 (2004). 2, 3

[18]

{Osi07} Osipa J.: Stop Staring. Sybex, 2007. 3

[19]

{PKC*03} Pyun H., Kim Y., Chae W., Kang H. W., Shin S. Y.: An example-based approach for facial expression cloning. In Proceedings of the ACM SIGGRAPH/Eurographics Symposium on Computer animation (2003), pp. 167--176. 3

Digital Library

[20]

{PL06} Pighin F., Lewis J. P. (Eds.):. Performance-Driven Facial Animation (2006), SIGGRAPH Course Notes. 2, 3

Digital Library

[21]

{Plu80} Plutchik R.: Emotion, a psychoevolutionary synthesis. Harper & Row (1980). 5

[22]

{PW96} Parke F. I., Waters K.: Computer Facial Animation. A K Peters, 1996. 2, 3

Digital Library

[23]

{SP04} Sumner R. W., Popovic J.: Deformation transfer for triangle meshes. ACM Transactions on Graphics 23, 3 (2004). 3

Digital Library

[24]

{VBPP05} Vlasic D., Brand M., Pfister H., Popovic J.: Face transfer with multilinear models. ACM Transactions on Graphics 24, 3 (2005), 426--433. 3

Digital Library

[25]

{WLGP09} Weise T., Li H., Gool L. V., Pauly M.: Face/off. Proceedings of the ACM SIGGRAPH/Eurographics Symposium on Computer Animation (2009). 2, 3

Digital Library

[26]

{ZLT*06} Zhang Q., Liu Z., Terzopoulos D., Guo B., Shum H.: Geometry-driven photorealistic facial expression synthesis. IEEE Transactions on Visualization and Computer Graphics 12, 1 (2006), 48--60. 3

Digital Library

[27]

{ZWT09} Zhang W., Wang Q., Tang X.: Performance driven face animation via non-rigid 3d tracking. Proceedings of the 17th ACM International Conference on Multimedia (2009). 3

Digital Library

Cited By

Zhu CJoslin C(2024)A review of motion retargeting techniques for 3D character facial animationComputers & Graphics10.1016/j.cag.2024.104037123(104037)Online publication date: Oct-2024
https://doi.org/10.1016/j.cag.2024.104037
Zell EMcDonnell R(2022)Compact Facial Landmark Layouts for Performance CaptureComputer Graphics Forum10.1111/cgf.1446341:2(121-133)Online publication date: 24-May-2022
https://doi.org/10.1111/cgf.14463
Thomas PCzerwinksi MMcduff DCraswell N(2021)Theories of Conversation for Conversational IRACM Transactions on Information Systems10.1145/343986939:4(1-23)Online publication date: 16-Aug-2021
https://dl.acm.org/doi/10.1145/3439869
Show More Cited By

Index Terms

Content retargeting using parameter-parallel facial layers
1. Computing methodologies
  1. Computer graphics
    1. Animation

Recommendations

Subtle facial expression recognition using motion magnification

This paper proposes a novel method for subtle facial expression recognition that uses motion magnification to transform subtle expressions into corresponding exaggerated ones. Motion magnification consists of four steps: First, active appearance model (...
Semantic 3D motion retargeting for facial animation
APGV '06: Proceedings of the 3rd symposium on Applied perception in graphics and visualization

We present a system for realistic facial animation that decomposes facial motion capture data into semantically meaningful motion channels based on the Facial Action Coding System. A captured performance is retargeted onto a morphable 3D face model ...
Facial expression recognition using 3D facial feature distances
ICIAR'07: Proceedings of the 4th international conference on Image Analysis and Recognition

In this paper, we propose a novel approach for facial expression analysis and recognition. The proposed approach relies on the distance vectors retrieved from 3D distribution of facial feature points to classify universal facial expressions. Neural ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

SCA '11: Proceedings of the 2011 ACM SIGGRAPH/Eurographics Symposium on Computer Animation

August 2011

297 pages

ISBN:9781450309233

DOI:10.1145/2019406

Conference Chairs:
Karen Liu
Georgia Institute of Technology
,
Robert Bridson
University of British Columbia
,
Editor:
Stephen N. Spencer
University of Washington
,
Program Chairs:
Adam Bargteil
University of Utah
,
Michiel van de Panne
University of British Columbia

Copyright © 2011 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

SIGGRAPH: ACM Special Interest Group on Computer Graphics and Interactive Techniques
EUROGRAPHICS: The European Association for Computer Graphics

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 05 August 2011

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Qualifiers

Research-article

Funding Sources

Division of Information and Intelligent Systems

Conference

SCA '11

Sponsor:

SIGGRAPH
EUROGRAPHICS

SCA '11: The ACM SIGGRAPH / Eurographics Symposium on Computer Animation 2011

August 5 - 7, 2011

British Columbia, Vancouver, Canada

Acceptance Rates

Overall Acceptance Rate 183 of 487 submissions, 38%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

15
Total Citations
View Citations
548
Total Downloads

Downloads (Last 12 months)4
Downloads (Last 6 weeks)0

Reflects downloads up to 13 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Zhu CJoslin C(2024)A review of motion retargeting techniques for 3D character facial animationComputers & Graphics10.1016/j.cag.2024.104037123(104037)Online publication date: Oct-2024
https://doi.org/10.1016/j.cag.2024.104037
Zell EMcDonnell R(2022)Compact Facial Landmark Layouts for Performance CaptureComputer Graphics Forum10.1111/cgf.1446341:2(121-133)Online publication date: 24-May-2022
https://doi.org/10.1111/cgf.14463
Thomas PCzerwinksi MMcduff DCraswell N(2021)Theories of Conversation for Conversational IRACM Transactions on Information Systems10.1145/343986939:4(1-23)Online publication date: 16-Aug-2021
https://dl.acm.org/doi/10.1145/3439869
McDuff DNowara EKitamura YQuigley AIsbister KIgarashi TBjørn PDrucker S(2021)“Warm Bodies”: A Post-Processing Technique for Animating Dynamic Blood Flow on Photos and AvatarsProceedings of the 2021 CHI Conference on Human Factors in Computing Systems10.1145/3411764.3445719(1-9)Online publication date: 6-May-2021
https://dl.acm.org/doi/10.1145/3411764.3445719
Ouzounis CKilias AMousas CJaillet FZara F(2017)Kernel projection of latent structures regression for facial animation retargetingProceedings of the 13th Workshop on Virtual Reality Interactions and Physical Simulations10.2312/vriphys.20171084(59-65)Online publication date: 23-Apr-2017
https://dl.acm.org/doi/10.2312/vriphys.20171084
Ribera RZell ELewis JNoh JBotsch M(2017)Facial retargeting with automatic range of motion alignmentACM Transactions on Graphics10.1145/3072959.307367436:4(1-12)Online publication date: 20-Jul-2017
https://dl.acm.org/doi/10.1145/3072959.3073674
Upchurch PGardner JPleiss GPless RSnavely NBala KWeinberger K(2017)Deep Feature Interpolation for Image Content Changes2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)10.1109/CVPR.2017.645(6090-6099)Online publication date: Jul-2017
https://doi.org/10.1109/CVPR.2017.645
Barrielle VStoiber NCagniart CGomes APurgathofer WJorge JLin M(2016)BlendForcesProceedings of the 37th Annual Conference of the European Association for Computer Graphics10.5555/3058909.3058955(341-352)Online publication date: 9-May-2016
https://dl.acm.org/doi/10.5555/3058909.3058955
Barrielle VStoiber NCagniart C(2016)BlendForces: A Dynamic Framework for Facial AnimationComputer Graphics Forum10.1111/cgf.1283635:2(341-352)Online publication date: 27-May-2016
https://doi.org/10.1111/cgf.12836
Liu SYang XWang ZXiao ZZhang J(2016)Real-time facial expression transfer with single video cameraComputer Animation and Virtual Worlds10.1002/cav.169727:3-4(301-310)Online publication date: 1-May-2016
https://dl.acm.org/doi/10.1002/cav.1697
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Table of Contents