research-article

Hybrid Skeletal-Surface Motion Graphs for Character Animation from 4D Performance Capture

Authors:

Margara Tejera,

John Collomosse,

Adrian HiltonAuthors Info & Claims

ACM Transactions on Graphics (TOG), Volume 34, Issue 2

Article No.: 17, Pages 1 - 14

https://doi.org/10.1145/2699643

Published: 02 March 2015 Publication History

Abstract

We present a novel hybrid representation for character animation from 4D Performance Capture (4DPC) data which combines skeletal control with surface motion graphs. 4DPC data are temporally aligned 3D mesh sequence reconstructions of the dynamic surface shape and associated appearance from multiple-view video. The hybrid representation supports the production of novel surface sequences which satisfy constraints from user-specified key-frames or a target skeletal motion. Motion graph path optimisation concatenates fragments of 4DPC data to satisfy the constraints while maintaining plausible surface motion at transitions between sequences. Space-time editing of the mesh sequence using a learned part-based Laplacian surface deformation model is performed to match the target skeletal motion and transition between sequences. The approach is quantitatively evaluated for three 4DPC datasets with a variety of clothing styles. Results for key-frame animation demonstrate production of novel sequences that satisfy constraints on timing and position of less than 1% of the sequence duration and path length. Evaluation of motion-capture-driven animation over a corpus of 130 sequences shows that the synthesised motion accurately matches the target skeletal motion. The combination of skeletal control with the surface motion graph extends the range and style of motion which can be produced while maintaining the natural dynamics of shape and appearance from the captured performance.

Supplementary Material

huang (huang.zip)

Supplemental movie, appendix, image and software files for, Hybrid Skeletal-Surface Motion Graphs for Character Animation from 4D Performance Capture

Download
114.19 MB

References

[1]

O. Arikan and D. Forsyth. 2002. Interactive motion generation from examples. ACM Trans. Graph. 21, 3, 483--490.

Digital Library

[2]

O. Arikan, D. Forsyth, and J. O'Brien. 2003. Motion synthesis from annotations. ACM Trans. Graph. 22, 3, 402--408.

Digital Library

[3]

M. Botsch and O. Sorkine. 2008. On linear variational surface deformation methods. IEEE Trans. Vis. Comput. Graph. 14, 1, 213--230.

Digital Library

[4]

C. Bregler, M. Covell, and M. Slaney. 1997. Video rewrite: Driving visual speech with audio. In Proceedings of the 24th Annual Conference on Computer Graphics and Interactive Techniques (SIGGRAPH'97). ACM, New York, NY, USA, 353--360.

Digital Library

[5]

C. Budd, P. Huang, M. Klaudiny, and A. Hilton. 2012. Global non-rigid alignment of surface sequences. Int. J. Comput. Vis. 102, 1--3, 256--270.

Digital Library

[6]

D. Casas, M. Tejera, J.-Y. Guillemaut, and A. Hilton. 2013. Interactive animation of 4D performance capture. IEEE Trans. Vis. Comput. Graph. 19, 5, 762--73.

Digital Library

[7]

D. Casas, M. Volino, J. Collomosse, and A. Hilton. 2014. 4D video textures for interactive character appearance. Comput. Graph. Forum 33, 2, 371--380.

Digital Library

[8]

G. L. Cmu. Carnegie mellon university motion capture database. http://mocap.cs.cmu.edu/.

[9]

T. Ezzat and G. Geiger. 2002. Trainable videorealistic speech animation. ACM Trans. Graph. 21, 3, 388--398.

Digital Library

[10]

M. Flagg, A. Nakazawa, Q. Zhang, S. B. Kang, Y. K. Ryu, I. Essa, and J. M. Rehg. 2009. Human video textures. In Proceedings of the Symposium on Interactive 3D Graphics and Games (I3D'09). ACM, New York, NY, USA, 199--206.

Digital Library

[11]

J. Gall, C. Stoll, and E. D. Aguiar. 2009. Motion capture using joint skeleton tracking and surface estimation. In Proceeding of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR'09). 1746--1753.

[12]

M. Gleicher, H. J. Shin, L. Kovar, and A. Jepsen. 2003. Snap-together motion: Assembling run-time animations. ACM Trans. Graph. 22, 3, 181--188.

Digital Library

[13]

R. Heck and M. Gleicher. 2007. Parametric motion graphs. In Proceedings of the Symposium on Interactive 3D Graphics and Games (I3D'07). ACM Press, New York, 129--136.

Digital Library

[14]

A. Hornung and E. Dekkers. 2007. Character animation from 2D pictures and 3D motion data. ACM Trans. Graph. 26, 1, 1--9.

Digital Library

[15]

P. Huang, A. Hilton, and J. Starck. 2009. Human motion synthesis from 3D video. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR'09). 1478--1485.

[16]

P. Huang, A. Hilton, and J. Starck. 2010. Shape +. Int. J. Comput. Vis. 89, 2--3, 362--381.

Digital Library

[17]

A. J. Hunt and A. W. Black. 1996. Unit selection in a concatenative speech synthesis system. In Proceedings of the IEEE International Conference on Acoustics, Speech, and Signal Processing (ICASSP'96). 373--376.

Digital Library

[18]

D. L. James, C. D. Twigg, A. Cove, and R. Y. Wang. 2007. Mesh ensemble motion graphs. ACM Trans. Graph. 26, 4.

Digital Library

[19]

S. Kircher and M. Garland. 2008. Free-form motion processing. ACM Trans. Graph. 27, 2, 1--13.

Digital Library

[20]

L. Kovar, M. Gleicher, and F. Pighin. 2002. Motion graphs. ACM Trans. Graph. 21, 3, 473--482.

Digital Library

[21]

A. Laurentini. 1994. The visual hull concept for silhouette-based image understanding. IEEE Trans. Pattern Anal. Mach. Intell. 16, 2, 150--162.

Digital Library

[22]

J. Lee, J. Chai, P. S. A. Reitsma, J. K. Hodgins, and N. S. Pollard. 2002. Interactive control of avatars animated with human motion data. ACM Trans. Graph. 21, 3, 491--500.

Digital Library

[23]

Y. Rubner, C. Tomasi, and L. Guibas. 1998. A metric for distributions with applications to image databases. In Proceedings of the IEEE International Conference on Computer Vision (ICCV'98). 59--66.

Digital Library

[24]

A. Schodl and I. Essa. 2002. Controlled animation of video sprites. In Proceedings of the ACM SIGGRAPH/Eurographics Symposium on Computer Animation (SCA'02). ACM Press, New York, 121--127.

Digital Library

[25]

A. Schodl, R. Szeliski, D. Salesin, and I. Essa. 2000. Video textures. In Proceedings of the 27th Annual Conference on Computer Graphics and Interactive Techniques (SIGGRAPH'00). ACM Press, 489--498.

Digital Library

[26]

O. Sorkine, D. Cohen-Or, Y. Lipman, M. Alexa, C. Rossl, and H.-P. Seidel. 2004. Laplacian surface editing. In Proceedings of the Eurographics/ACM SIGGRAPH Symposium on Geometry Processing (SGP'04). ACM Press, New York, 175--184.

Digital Library

[27]

J. Starck and A. Hilton. 2007. Surface capture for performance-based animation. IEEE Comput. Graph. Appl. 27, 3, 21--31.

Digital Library

[28]

J. Starck, G. Miller, and A. Hilton. 2005. Video-based character animation. In Proceedings of the ACM SIGGRAPH/Eurographics Symposium on Computer Animation (SCA'05). ACM Press, New York, 49--58.

Digital Library

[29]

R. W. Sumner, M. Zwicker, C. Gotsman, and J. Popovic. 2005. Mesh-based inverse kinematics. ACM Trans. Graph. 24, 3, 488--495.

Digital Library

[30]

M. Tejera, D. Casas, and A. Hilton. 2013. Animation control of surface motion capture. IEEE Trans. Cybernet. 43, 1532--1545.

[31]

M. Tejera and A. Hilton. 2013. Learning part-based models for animation from surface motion capture. In Proceedings of the International Conference on 3D Vision (3DV'13). 159--166.

Digital Library

[32]

F. Xu, Y. Liu, J. Tompkin, G. Bharaj, J. Kautz, C. Theobalt, C. Stoll, Q. Dai, and H.-P. Seidel. 2011. Video-based characters creating new human performances from a multi-view video database. ACM Trans. Graph. 30, 4, 32:1--32:10.

Digital Library

[33]

W. Xu, K. Zhou, Y. Yu, Q. Tan, Q. Peng, and B. Guo. 2007. Gradient domain editing of deforming mesh sequences. ACM Trans. Graph. 26, 3.

Digital Library

Cited By

Cheon EXu S(2024)Creative Precarity in Motion: Revealing the Hidden Labor Behind Animating Virtual CharactersProceedings of the 2024 ACM Designing Interactive Systems Conference10.1145/3643834.3661545(3471-3484)Online publication date: 1-Jul-2024
https://dl.acm.org/doi/10.1145/3643834.3661545
Deng XZheng ZZhang YSun JXu CYang XWang LLiu Y(2024)RAM-Avatar: Real-time Photo-Realistic Avatar from Monocular Videos with Full-body Control2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)10.1109/CVPR52733.2024.00195(1996-2007)Online publication date: 16-Jun-2024
https://doi.org/10.1109/CVPR52733.2024.00195
Hácha FDvořák JKáčereková ZVáša L(2024)Editing mesh sequences with varying connectivityComputers & Graphics10.1016/j.cag.2024.103943121(103943)Online publication date: Jun-2024
https://doi.org/10.1016/j.cag.2024.103943
Show More Cited By

Index Terms

Hybrid Skeletal-Surface Motion Graphs for Character Animation from 4D Performance Capture
1. Computing methodologies
  1. Computer graphics
    1. Animation

Recommendations

4D parametric motion graphs for interactive animation
I3D '12: Proceedings of the ACM SIGGRAPH Symposium on Interactive 3D Graphics and Games

A 4D parametric motion graph representation is presented for interactive animation from actor performance capture in a multiple camera studio. The representation is based on a 4D model database of temporally aligned mesh sequence reconstructions for ...
Interactive Animation of 4D Performance Capture

A 4D parametric motion graph representation is presented for interactive animation from actor performance capture in a multiple camera studio. The representation is based on a 4D model database of temporally aligned mesh sequence reconstructions for ...
Learning Part-Based Models for Animation from Surface Motion Capture
3DV '13: Proceedings of the 2013 International Conference on 3D Vision

Surface motion capture (Surf Cap) enables 3D reconstruction of human performance with detailed cloth and hair deformation. However, there is a lack of tools that allow flexible editing of Surf Cap sequences. In this paper, we present a Laplacian editing ...

Comments

Information & Contributors

Information

Published In

cover image ACM Transactions on Graphics

ACM Transactions on Graphics Volume 34, Issue 2

February 2015

136 pages

ISSN:0730-0301

EISSN:1557-7368

DOI:10.1145/2742222

Editor:
Holly Rushmeier
Yale University

Issue’s Table of Contents

Copyright © 2015 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 02 March 2015

Accepted: 01 September 2014

Revised: 01 February 2014

Received: 01 March 2013

Published in TOG Volume 34, Issue 2

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed

Funding Sources

EU-FP7 project RE@CT
EPSRC Platform

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

38
Total Citations
View Citations
727
Total Downloads

Downloads (Last 12 months)17
Downloads (Last 6 weeks)0

Reflects downloads up to 21 Sep 2024

Other Metrics

View Author Metrics

Citations

Cited By

Cheon EXu S(2024)Creative Precarity in Motion: Revealing the Hidden Labor Behind Animating Virtual CharactersProceedings of the 2024 ACM Designing Interactive Systems Conference10.1145/3643834.3661545(3471-3484)Online publication date: 1-Jul-2024
https://dl.acm.org/doi/10.1145/3643834.3661545
Deng XZheng ZZhang YSun JXu CYang XWang LLiu Y(2024)RAM-Avatar: Real-time Photo-Realistic Avatar from Monocular Videos with Full-body Control2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)10.1109/CVPR52733.2024.00195(1996-2007)Online publication date: 16-Jun-2024
https://doi.org/10.1109/CVPR52733.2024.00195
Hácha FDvořák JKáčereková ZVáša L(2024)Editing mesh sequences with varying connectivityComputers & Graphics10.1016/j.cag.2024.103943121(103943)Online publication date: Jun-2024
https://doi.org/10.1016/j.cag.2024.103943
Ma XZhao JTeng YYao L(2023)Multi‐Level Implicit Function for Detailed Human Reconstruction by Relaxing SMPL ConstraintsComputer Graphics Forum10.1111/cgf.1495142:7Online publication date: 30-Oct-2023
https://doi.org/10.1111/cgf.14951
Harris EKhoo IDemircan E(2022)A Survey of Human Gait-Based Artificial Intelligence ApplicationsFrontiers in Robotics and AI10.3389/frobt.2021.7492748Online publication date: 3-Jan-2022
https://doi.org/10.3389/frobt.2021.749274
Zhou YYang JLi DSaito JAneja DKalogerakis E(2022)Audio-driven Neural Gesture Reenactment with Video Motion Graphs2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)10.1109/CVPR52688.2022.00341(3408-3418)Online publication date: Jun-2022
https://doi.org/10.1109/CVPR52688.2022.00341
Regateiro JVolino MHilton A(2021)Deep4D: A Compact Generative Representation for Volumetric VideoFrontiers in Virtual Reality10.3389/frvir.2021.7390102Online publication date: 1-Nov-2021
https://doi.org/10.3389/frvir.2021.739010
Ye F(2021)[Retracted] Application of Photoshop Graphics and Image Processing in the Field of AnimationJournal of Sensors10.1155/2021/86774792021:1Online publication date: 18-Oct-2021
https://doi.org/10.1155/2021/8677479
Wu JJin YGeng ZZhou HFedkiw R(2021)Recovering Geometric Information with Learned Texture PerturbationsProceedings of the ACM on Computer Graphics and Interactive Techniques10.1145/34801374:3(1-18)Online publication date: 27-Sep-2021
https://dl.acm.org/doi/10.1145/3480137
MacQuarrie ASteed A(2020)Exploring the Use of Skeletal Tracking for Cheaper Motion Graphs and On-Set Decision Making in Free-Viewpoint Video ProductionProceedings of the 17th ACM SIGGRAPH European Conference on Visual Media Production10.1145/3429341.3429353(1-10)Online publication date: 7-Dec-2020
https://dl.acm.org/doi/10.1145/3429341.3429353
Show More Cited By

View Options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Media

Figures

Other

Tables

View Issue’s Table of Contents