research-article

Animation cartography—intrinsic reconstruction of shape and motion

Authors:

Alexander Berner,

Martin Bokeloh,

Hans-Peter SeidelAuthors Info & Claims

ACM Transactions on Graphics (TOG), Volume 31, Issue 2

Article No.: 12, Pages 1 - 15

https://doi.org/10.1145/2159516.2159517

Published: 30 April 2012 Publication History

Abstract

In this article, we consider the problem of animation reconstruction, that is, the reconstruction of shape and motion of a deformable object from dynamic 3D scanner data, without using user-provided template models. Unlike previous work that addressed this problem, we do not rely on locally convergent optimization but present a system that can handle fast motion, temporally disrupted input, and can correctly match objects that disappear for extended time periods in acquisition holes due to occlusion. Our approach is motivated by cartography: We first estimate a few landmark correspondences, which are extended to a dense matching and then used to reconstruct geometry and motion. We propose a number of algorithmic building blocks: a scheme for tracking landmarks in temporally coherent and incoherent data, an algorithm for robust estimation of dense correspondences under topological noise, and the integration of local matching techniques to refine the result. We describe and evaluate the individual components and propose a complete animation reconstruction pipeline based on these ideas. We evaluate our method on a number of standard benchmark datasets and show that we can obtain correct reconstructions in situations where other techniques fail completely or require additional user guidance such as a template model.

Supplementary Material

JPG File (tp136_12.jpg)

Download
18.15 KB

tevs (tevs.zip)

Supplemental movie and image files for, Animation cartography—intrinsic reconstruction of shape and motion

Download
283.56 MB

MP4 File (tp136_12.mp4)

Download
39.53 MB

References

[1]

Ahmed, N., Theobalt, C., Rossl, C., Thrun, S., and Seidel, H.-P. 2008. Dense correspondence finding for parameterization-free animation reconstruction from video. In Proceedings of the Conference on Computer Vision and Pattern Recognition (CVPR'08). 1--8.

[2]

Allen, B., Curless, B., and Popović, Z. 2003. The space of human body shapes: Reconstruction and parameterization from range scans. ACM Trans. Graph. 22, 3.

Digital Library

[3]

Anguelov, D., Srinivasan, P., Pang, H.-C., Koller, D., Thrun, S., and Davis, J. 2004. The correlated correspondence algorithm for unsupervised registration of nonrigid surfaces. In Proceedings of the Conference on Neural Information Processing Systems (NIPS'04).

[4]

Anuar, N. and Guskov, I. 2004. Extracting animated meshes with adaptive motion estimation. In Proceedings of the Vision, Modeling, and Visualization Workshop (VMV'04). 63--71.

[5]

Bickel, B., Botsch, M., Angst, R., Matusik, W., Otaduy, M., Pfister, H., and Gross, M. 2007. Multi-Scale capture of facial geometry and motion. ACM Trans. Graph. 26, 3, 33.

Digital Library

[6]

Bradley, D., Heidrich, W., Popa, T., and Sheffer, A. 2010. High resolution passive facial performance capture. ACM Trans. Graph. 29, 41:1-41:10.

Digital Library

[7]

Bradley, D., Popa, T., Sheffer, A., Heidrich, W., and Boufekeur, T. 2008. Markerless garment capture. ACM Trans. Graph. 27, 3, 99.

Digital Library

[8]

Bronstein, A. M., Bronstein, M. M., and Kimmel, R. 2006. Generalized multidimensional scaling: A framework for isometry-invariant partial surface matching. Proc. Nat. Acad. Sci. 103, 5, 1168--1172.

[9]

Bronstein, A. M., Bronstein, M. M., and Kimmel, R. 2009. Topology-Invariant similarity of nonrigid shapes. Int. J. Comput. Vis. 81, 3, 281--301.

Digital Library

[10]

Bronstein, A. M., Bronstein, M. M., Kimmel, R., Mahmoudi, M., and Sapiro, G. 2010. A gromov-hausdorff framework with diffusion geometry for topologically-robust nonrigid shape matching. Int. J. Comput. Vis. 89, 266--286.

Digital Library

[11]

Carranza, J., Theobalt, C., Magnor, M. A., and Seidel, H.-P. 2003. Free-Viewpoint video of human actors. ACM Trans. Graph. 22, 569--577.

Digital Library

[12]

Chang, W. and Zwicker, M. 2008. Automatic registration for articulated shapes. Comput. Graph. Forum 27, 5, 1459--1468.

Digital Library

[13]

Chang, W. and Zwicker, M. 2009. Range scan registration using reduced deformable models. Comput. Graph. Forum 28, 2, 447--456.

[14]

Chum, O. and Matas, J. 2005. Matching with PROSAC—Progressive sample consensus. In Proceedings of the Conference on Computer Vision and Pattern Recognition (CVPR'05). Vol. 1. 220--226.

Digital Library

[15]

Davis, J., Nehab, D., Ramamoorthi, R., and Rusinkiewicz, S. 2005. Spacetime stereo: A unifying framework for depth from triangularization. IEEE Trans. Pattern Anal. Mach. Intell. 27, 2, 296--302.

Digital Library

[16]

De Aguiar, E., Stoll, C., Theobalt, C., Ahmed, N., Seidel, H.-P., and Thrun, S. 2008. Performance capture from sparse multi-view video. ACM Trans. Graph. 27, 3, 98.

Digital Library

[17]

Fang, Q., Gao, J., Guibas, L. J., de Silva, V., and Zhang, L. 2005. Glider: Gradient landmark-based distributed routing for sensor networks. In Proceedings of the 24^th Conference of the IEEE Communications Society (InfoCom).

[18]

Gelfand, N. and Guibas, L. J. 2004. Shape segmentation using local slippage analysis. In Proceedings of the Symposium on Geometry Processing (SGP'04). 214--223.

Digital Library

[19]

Häehnel, D., Thrun, S., and Burgard, W. 2003. An extension of the icp algorithm for modeling nonrigid objects with mobile robots. In Proceedings of the International Joint Conference on Artificial Intelligence (IJCAI). 915--920.

Digital Library

[20]

Horn, B. K. P. 1987. Closed-Form solution of absolute orientation using unit quaternions. J. Opt. Soc. Amer. A4, 4, 629--642.

[21]

Huang, Q.-X., Adams, B., Wicke, M., and Guibas, L. J. 2008. Nonrigid registration under isometric deformations. Comput. Graph. Forum 27, 5, 1449--1457.

Digital Library

[22]

Huang, Q.-X., Flöry, S., Gelfand, N., Hofer, M., and Pottmann, H. 2006. Reassembling fractured objects by geometric matching. ACM Trans. Graph. 25, 3, 569--578.

Digital Library

[23]

König, S. and Gumhold, S. 2008. Image-Based motion compensation for structured light scanning of dynamic scenes. Int. J. Int. Syst. Tech. Appl. 5, 3-4, 434--441.

Digital Library

[24]

Leordenu, M. and Hebert, M. 2005. A spectral technique for correspondence problems using pairwise constraints. In Proceedings of the International Conference on Computer Vision (ICCV'05). Vol. 2, 1482--1489.

Digital Library

[25]

Li, H., Adams, B., Guibas, L. J., and Pauly, M. 2009. Robust single-view geometry and motion reconstruction. ACM Trans. Graph. 28, 5, 175.

Digital Library

[26]

Li, H., Sumner, R. W., and Pauly, M. 2008. Global correspondence optimization for nonrigid registration of depth scans. Comput. Graph. Forum 27, 5, 1421--1430.

Digital Library

[27]

Liao, M., Zhang, Q., Wang, H., Yang, R., and Gong, M. 2009. Modeling deformable objects from a single depth camera. In Proceedings of the International Conference on Computer Vision (ICCV'09).

[28]

Lipman, Y. and Funkhouser, T. 2009. Möbius voting for surface correspondence. In Proceedings of the SIGGRAPH'09 Conference. ACM Press, New York, 1--12.

Digital Library

[29]

Lucas, B. D. and Kanade, T. 1981. An iterative image registration technique with an application to stereo vision. In Proceedings of the International Joint Conference on Artificial Intelligence (IJCAI'81). 674--679.

Digital Library

[30]

Mitra, N. J., Flory, S., Ovsjanikov, M., Gelfand, N., Guibas, L., and Pottsmann, H. 2007. Dynamic geometry registration. In Proceedings of the Symposium on Geometry Processing (SGP'07). 173--182.

Digital Library

[31]

Ovsjanikov, M., Mérigot, Q., Mémoli, F., and Guibas, L. 2010. One point isometric matching with the heat kernel. In Proceedings of the Symposium on Geometry Processing (SGP'10). 1555--1564.

[32]

Park, S. I. and Hodgins, J. K. 2006. Capturing and animating skin deformation in human motion. ACM Trans. Graph. 25, 3, 881--889.

Digital Library

[33]

Pekelny, Y. and Gotsman, C. 2008. Articulated object reconstruction and markerless motion capture from depth video. Comput. Graph. Forum 27, 2, 399--408.

[34]

Popa, T., South-Dickenson, I., Bradley, D., Sheffer, A., and Heidrich, W. 2010. Globally consistent space-time reconstruction. Comput. Graph. Forum 29, 5, 1633--1642.

[35]

Sand, P., McMillan, L., and Popović, J. 2003. Continuous capture of skin deformation. ACM Trans. Graph. 22, 3, 578--586.

Digital Library

[36]

Sharf, A., Alcantara, D. A., Lewiner, T., Greif, C., Sheffer, A., Amenta, N., and Cohen-Or, D. 2008. Space-Time surface reconstruction using incompressible flow. ACM Trans. Graph. 27, 5, 110.

Digital Library

[37]

Starck, J. and Hilton, A. 2007. Correspondence labeling for wide-timeframe free-form surface matching. In Proceedings of the International Conference on Computer Vision (ICCV'07). 1--8.

[38]

Süssmuth, J., Winter, M., and Greiner, G. 2008. Reconstructing animated meshes from time-varying point clouds. Comput. Graph. Forum 27, 5, 1469--1476.

Digital Library

[39]

Tevs, A., Berner, A., Wand, M., Ihrke, I., and Seidel, H.-P. 2011. Intrinsic shape matching by planned landmark sampling. Comput. Graph. Forum 30, 543--552.

[40]

Tevs, A., Bokeloh, M., Wand, M., Schilling, A., and Seidel, H.-P. 2009. Isometric registration of ambiguous and partial data. In Proceedings of the Conference on Computer Vision and Pattern Recognition (CVPR'09). 1185--1192.

[41]

Varanasi, K., Zaharescu, A., Boyer, E., and Horaud, R. P. 2008. Temporal surface tracking using mesh evolution. In Proceedings of the European Conference on Computer Vision (ECCV'08). 30--43.

Digital Library

[42]

Vlasic, D., Peers, P., Baran, I., Debevec, P., Popović, J., Rusinkiewicz, S., and Matusik, W. 2009. Dynamic shape capture using multi-view photometric stereo. ACM Trans. Graph. 28, 5, 174.

Digital Library

[43]

Wand, M., Adams, B., Ovsjanikov, M., Berner, A., Bokeloh, M., Jenke, P., Guibas, L., Seidel, H.-P., and Schilling, A. 2009. Efficient reconstruction of nonrigid shape and motion from real-time 3D scanner data. ACM Trans. Graph. 28, 2, 1--15.

Digital Library

[44]

Wand, M., Jenke, P., Huang, Q., Bokeloh, M., Guibas, L., and Schilling, A. 2007. Reconstruction of deforming geometry from time-varying point clouds. In Proceedings of the Symposium on Geometric Processing (SGP'07). 49--58.

Digital Library

[45]

Weise, T., Leife, B., and Gool, L. V. 2007. Fast 3D scanning with automatic motion compensation. In Proceedings of the Conference on Computer Vision and Pattern Recognition (CVPR'07). 1--8.

[46]

Würmlin, S., Lamboray, E., Staadt, O. G., and Gross, M. H. 2002. 3D video recorder. In Proceedings of the 10^th Pacific Conference on Computer Graphics and Applications (PG'02). IEEE Computer Society, Washington, DC, 325.

Digital Library

[47]

Zhang, L., Snavely, N., Curless, B., and Seitz, S. M. 2004. Space-Time faces: High resolution capture for modeling and animation. ACM Trans. Graph. 23, 3, 548--558.

Digital Library

[48]

Zheng, Q., Sharf, A., Tagliasacchi, A., Chen, B., Zhang, H., Sheffer, A., and Cohen-Or, D. 2010. Consensus skeleton for nonrigid space-time registration. Comput. Graph. Forum 29, 2, 635--644.

[49]

Zitnick, C. L., Kang, S. B., Uyttendaele, M., Winder, S., and Szeliski, R. 2004. High-Quality video view interpolation using a layered representation. In ACM SIGGRAPH'04 Papers. ACM, New York, 600--608.

Digital Library

Cited By

Bai YTran HDamelin SKolouri S(2025)Partial transport for point-cloud registrationSampling Theory, Signal Processing, and Data Analysis10.1007/s43670-025-00097-123:1Online publication date: 18-Feb-2025
https://doi.org/10.1007/s43670-025-00097-1
Hácha FDvořák JKáčereková ZVáša L(2024)Editing mesh sequences with varying connectivityComputers and Graphics10.1016/j.cag.2024.103943121:COnline publication date: 1-Jun-2024
https://dl.acm.org/doi/10.1016/j.cag.2024.103943
Su ZXu LZhong DLi ZDeng FQuan SFang L(2023)RobustFusion: Robust Volumetric Performance Reconstruction Under Human-Object Interactions From Monocular RGBD StreamIEEE Transactions on Pattern Analysis and Machine Intelligence10.1109/TPAMI.2022.321574645:5(6196-6213)Online publication date: 1-May-2023
https://dl.acm.org/doi/10.1109/TPAMI.2022.3215746
Show More Cited By

Index Terms

Animation cartography—intrinsic reconstruction of shape and motion
1. Computing methodologies
  1. Artificial intelligence
    1. Computer vision
      1. Computer vision problems
      2. Computer vision tasks
        Scene understanding
  2. Computer graphics
    1. Animation
    2. Shape modeling
      1. Parametric curve and surface models

Recommendations

Robust single-view geometry and motion reconstruction

We present a framework and algorithms for robust geometry and motion reconstruction of complex deforming shapes. Our method makes use of a smooth template that provides a crude approximation of the scanned object and serves as a geometric and topological ...
Efficient reconstruction of nonrigid shape and motion from real-time 3D scanner data

We present a new technique for reconstructing a single shape and its nonrigid motion from 3D scanning data. Our algorithm takes a set of time-varying unstructured sample points that capture partial views of a deforming object as input and reconstructs a ...
Robust single-view geometry and motion reconstruction
SIGGRAPH Asia '09: ACM SIGGRAPH Asia 2009 papers

We present a framework and algorithms for robust geometry and motion reconstruction of complex deforming shapes. Our method makes use of a smooth template that provides a crude approximation of the scanned object and serves as a geometric and topological ...

Comments

Information & Contributors

Information

Published In

cover image ACM Transactions on Graphics

ACM Transactions on Graphics Volume 31, Issue 2

April 2012

78 pages

ISSN:0730-0301

EISSN:1557-7368

DOI:10.1145/2159516

Issue’s Table of Contents

Copyright © 2012 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 30 April 2012

Accepted: 01 October 2011

Revised: 01 August 2011

Received: 01 February 2011

Published in TOG Volume 31, Issue 2

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

68
Total Citations
View Citations
836
Total Downloads

Downloads (Last 12 months)12
Downloads (Last 6 weeks)0

Reflects downloads up to 25 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Bai YTran HDamelin SKolouri S(2025)Partial transport for point-cloud registrationSampling Theory, Signal Processing, and Data Analysis10.1007/s43670-025-00097-123:1Online publication date: 18-Feb-2025
https://doi.org/10.1007/s43670-025-00097-1
Hácha FDvořák JKáčereková ZVáša L(2024)Editing mesh sequences with varying connectivityComputers and Graphics10.1016/j.cag.2024.103943121:COnline publication date: 1-Jun-2024
https://dl.acm.org/doi/10.1016/j.cag.2024.103943
Su ZXu LZhong DLi ZDeng FQuan SFang L(2023)RobustFusion: Robust Volumetric Performance Reconstruction Under Human-Object Interactions From Monocular RGBD StreamIEEE Transactions on Pattern Analysis and Machine Intelligence10.1109/TPAMI.2022.321574645:5(6196-6213)Online publication date: 1-May-2023
https://dl.acm.org/doi/10.1109/TPAMI.2022.3215746
Laga HPadilla MJermyn IKurtek SBennamoun MSrivastava A(2023)4D Atlas: Statistical Analysis of the Spatiotemporal Variability in Longitudinal 3D Shape DataIEEE Transactions on Pattern Analysis and Machine Intelligence10.1109/TPAMI.2022.316372045:2(1335-1352)Online publication date: 1-Feb-2023
https://doi.org/10.1109/TPAMI.2022.3163720
Monji-Azad SHesser JLöw N(2023)A review of non-rigid transformations and learning-based 3D point cloud registration methodsISPRS Journal of Photogrammetry and Remote Sensing10.1016/j.isprsjprs.2022.12.023196(58-72)Online publication date: Feb-2023
https://doi.org/10.1016/j.isprsjprs.2022.12.023
Moschella LMelzi SCosmo LMaggioli FLitany OOvsjanikov MGuibas LRodolà E(2022)Learning Spectral Unions of Partial Deformable 3D ShapesComputer Graphics Forum10.1111/cgf.1448341:2(407-417)Online publication date: 24-May-2022
https://doi.org/10.1111/cgf.14483
Dvořák JKáčereková ZVaněček PHruda LVáša L(2022)As-rigid-as-possible volume tracking for time-varying surfacesComputers and Graphics10.1016/j.cag.2021.10.015102:C(329-338)Online publication date: 1-Feb-2022
https://dl.acm.org/doi/10.1016/j.cag.2021.10.015
Luo GDeng ZZhao XJin XZeng WXie WSeo H(2020)Spatio-temporal Segmentation Based Adaptive Compression of Dynamic Mesh SequencesACM Transactions on Multimedia Computing, Communications, and Applications10.1145/337747516:1(1-24)Online publication date: 4-Mar-2020
https://dl.acm.org/doi/10.1145/3377475
Innmann MKim KGu JNiesner MLoop CStamminger MKautz J(2020)NRMVS: Non-Rigid Multi-View Stereo2020 IEEE Winter Conference on Applications of Computer Vision (WACV)10.1109/WACV45572.2020.9093583(2743-2752)Online publication date: Mar-2020
https://doi.org/10.1109/WACV45572.2020.9093583
Yu TZhao JZheng ZGuo KDai QLi HPons-Moll GLiu Y(2020)DoubleFusion: Real-Time Capture of Human Performances with Inner Body Shapes from a Single Depth SensorIEEE Transactions on Pattern Analysis and Machine Intelligence10.1109/TPAMI.2019.292829642:10(2523-2539)Online publication date: 1-Oct-2020
https://doi.org/10.1109/TPAMI.2019.2928296
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Issue’s Table of Contents