research-article

Efficient reconstruction of nonrigid shape and motion from real-time 3D scanner data

Authors:

Maksim Ovsjanikov,

Alexander Berner,

Martin Bokeloh,

Leonidas Guibas,

Hans-Peter Seidel,

Andreas SchillingAuthors Info & Claims

ACM Transactions on Graphics (TOG), Volume 28, Issue 2

Article No.: 15, Pages 1 - 15

https://doi.org/10.1145/1516522.1516526

Published: 13 May 2009 Publication History

Abstract

We present a new technique for reconstructing a single shape and its nonrigid motion from 3D scanning data. Our algorithm takes a set of time-varying unstructured sample points that capture partial views of a deforming object as input and reconstructs a single shape and a deformation field that fit the data. This representation yields dense correspondences for the whole sequence, as well as a completed 3D shape in every frame. In addition, the algorithm automatically removes spatial and temporal noise artifacts and outliers from the raw input data. Unlike previous methods, the algorithm does not require any shape template but computes a fitting shape automatically from the input data. Our reconstruction framework is based upon a novel topology-aware adaptive subspace deformation technique that allows handling long sequences with complex geometry efficiently. The algorithm accesses data in multiple sequential passes, so that long sequences can be streamed from hard disk, not being limited by main memory. We apply the technique to several benchmark datasets, significantly increasing the complexity of the data that can be handled efficiently in comparison to previous work.

Supplementary Material

wand (wand.mov)

1st supplemental movie file for Efficient reconstruction of nonrigid shape and motion from real-time 3D scanner data

Download
65.60 MB

References

[1]

Adams, B., Ovsjanikov, M., Wand, M., Seidel, H.-P., and Guibas, L. J. 2008. Meshless modeling of deformable shapes and their motion. In ACM SIGGRAPH/Eurographics Symposium on Computer Animation.

Digital Library

[2]

Ahmed, N., Theobalt, C., Roessl, C., Thrun, S., and Seidel, H.-P. 2008. Dense correspondence finding for parametrization-free animation reconstruction from video. In Proceedings of the Conference on Computer Vision and Pattern Recognition (CVPR).

[3]

Allen, B., Curless, B., and Popović, Z. 2002. Articulated body deformation from range scan data. Proceedings of the ACM SIGGRAPH International Conference on Computer Graphics and Interactive Techniques. 612--619.

Digital Library

[4]

Allen, B., Curless, B., and Popović, Z. 2003. The space of human body shapes: Reconstruction and parameterization from range scans. In ACM SIGGRAPH Papers. ACM, New York, 587--594.

Digital Library

[5]

Anguelov, D., Koller, D., Srinivasan, P., Thrun, S., Pang, H.-C., and Davis, J. 2005. The correlated correspondence algorithm for unsupervised registration of nonrigid surfaces. In Proceedings of the Conference on Advances in Neural Information Processing Systems (NIPS ).

[6]

Anguelov, D., Srinivasan, P., Koller, D., Thrun, S., Rodgers, J., and Davis, J. 2005. SCAPE: Shape completion and animation of people. In Proceedings of the 32nd International Conference on Computer Graphics and Interactive Techniques (SIGGRAPH).

Digital Library

[7]

Anuar, N. and Guskov, I. 2004. Extracting animated meshes with adaptive motion estimation. In Proceedings of the Vision, Modeling and Visualization Conference. 63--71.

[8]

Au, O. K.-C., Tai, C.-L., Liu, L., and Fu, H. 2006. Dual laplacian editing for meshes. IEEE Trans. Visualiz. Comput. Graph. 12, 3, 386--395.

Digital Library

[9]

Avriel, M. 2003. Nonlinear Programming: Analysis and Methods. Dover Publishing.

[10]

Besl, P. J. and McKay, N. D. 1992. A method for registration of 3-d shapes. IEEE Trans. Pattern Anal. Mach. Intell. 14, 2, 239--256.

Digital Library

[11]

Botsch, M., Pauly, M., Gross, M., and Kobbelt, L. 2006. Primo: Coupled prisms for intuitive surface modeling. In SGP '06: Proceedings of the 4th Eurographics Symposium on Geometry Processing. Eurographics Association. 11--20.

Digital Library

[12]

Botsch, M., Pauly, M., Wicke, M., and Gross, M. 2007. Adaptive space deformations based on rigid cells. Comput. Graph. Forum 26, 3, 339--347.

[13]

Botsch, M. and Sorkine, O. 2008. On linear variational surface deformation methods. IEEE Trans. Visualiz. Comput. Graph. 14, 1, 213--230.

Digital Library

[14]

Carceroni, R. L. and Kutulakos, K. N. 2002. Multi-View scene capture by surfel sampling: From video streams to non-rigid 3d motion, shape and reflectance. Int. J. Comput. Vision 49, 2-3, 175--214.

Digital Library

[15]

Carranza, J., Theobalt, C., Magnor, M. A., and Seidel, H.-P. 2003. Free-Viewpoint video of human actors. In ACM SIGGRAPH Papers. ACM, New York, 569--577.

Digital Library

[16]

Chang, W. and Zwicker, M. 2008. Automatic registration for articulated shapes. Comput. Graph. Forum 27, 5.

Digital Library

[17]

Davis, J., Nehab, D., Ramamoorthi, R., and Rusinkiewicz, S. 2005. Spacetime stereo: A unifying framework for depth from triangulation. IEEE Trans. Pattern Anal. Mach. Intell. 27, 2, 296--302.

Digital Library

[18]

de Aguiar, E., Stoll, C., Theobalt, C., Ahmed, N., Seidel, H.-P., and Thrun, S. 2008. Performance capture from sparse multi-view video. ACM Trans. Graph. 27, 3, 1--10.

Digital Library

[19]

Fong, P. and Buron, F. 2005. High-Resolution three-dimensional sensing of fast deforming objects. In Proceedings of the IEEE/RSI International Conference on Intelligent Robots and Systems, (IROS). 1606--1611.

[20]

Fries, T.-P. and Matthies, H. G. 2003. Classification and overview of meshfree methods. Tech. rep., TU Brunswick, Germany Nr. 2003-03.

[21]

Huang, J., Shi, X., Liu, X., Zhou, K., Wei, L.-Y., Teng, S.-H., Bao, H., Guo, B., and Shum, H.-Y. 2006. Subspace gradient domain mesh deformation. ACM Trans. Graph. 25, 3, 1126--1134.

Digital Library

[22]

Huang, Q.-X., Adams, B., and Wand, M. 2007. Bayesian surface reconstruction via iterative scan alignment to an optimized prototype. In Proceedings of the 5th Eurographics Symposium on Geometry Processing (SCP). Eurographics Association, 213--223.

Digital Library

[23]

Huang, Q.-X., Adams, B., Wicke, M., and Guibas, L. J. 2008. Non-Rigid registration under isometric deformations. Comput. Graph. Forum 27, 5.

[24]

König, S. and Gumhold, S. 2007. Image-Based motion compensation for structured light scanning of dynamic scenes. In EG Workshop on Dynamic 3D Imaging (Dyn3D '07).

[25]

Li, H., Sumner, R. W., and Pauly, M. 2008. Global correspondence optimization for non-rigid registration of depth scans. Comput. Graph. Forum 27, 5.

Digital Library

[26]

Medioni, G., Lee, M., and Tang, C. 2000. A Computational Framework for Segmentation and Grouping. Elsevier.

Digital Library

[27]

MESA. 2009. http://www.mesa-imaging.ch.

[28]

Mitra, N. J., Flöry, S., Ovsjanikov, M., Gelfand, N., Guibas, L., and Pottmann, H. 2007. Dynamic geometry registration. In Proceedings of the 5th Eurographics Symposium on Geometry Processing (SGP '07). Eurographics Association, 173--182.

Digital Library

[29]

Müller, M., Keiser, R., Nealen, A., Pauly, M., Gross, M., and Alexa, M. 2004. Point based animation of elastic, plastic and melting objects. In Proceedings of the ACM SIGGRAPH/Eurographics Symposium on Computer Animation (SCA '04). Eurographics Association, 141--151.

Digital Library

[30]

Nealen, A., Müller, M., Keiser, R., Boxerman, E., and Carlson, M. 2005. Physically based deformable models in computer graphics. In Eurographics '05 STAR Reports, 71--94.

[31]

Park, S. I. and Hodgins, J. K. 2006. Capturing and animating skin deformation in human motion. ACM Trans. Graph. 25, 3, 881--889.

Digital Library

[32]

Pauly, M., Keiser, R., Adams, B., Dutré, P., Gross, M., and Guibas, L. J. 2005. Meshless animation of fracturing solids. ACM Trans. Graph. 24, 3, 957--964.

Digital Library

[33]

Pekelny, Y. and Gotsman, C. 2008. Articulated object reconstruction and markerless motion capture from depth video. Comput. Graph. Forum (Proceedings of the Eurographics EG'08) 27, 2, 4.

[34]

PMD. 2009. http://www.pmdtec.com.

[35]

Sand, P., McMillan, L., and Popović, J. 2003. Continuous capture of skin deformation. ACM Trans. Graph. 22, 3, 578--586.

Digital Library

[36]

Sharf, A., Alcantara, D. A., Lewiner, T., Greif, C., Sheffer, A., Amenta, N., and Cohen-Or, D. 2008. Space-Time surface reconstruction using incompressible flow. ACM Trans. Graph. (Proc. Siggraph Asia).

Digital Library

[37]

Sheffer, A. and Kraevoy, V. 2004. Pyramid coordinates for morphing and deformation. In Proceedings of the 3D Data Processing, Visualization, and Transmission, 2nd International Symposium on (3DPVT'04). IEEE Computer Society, 68--75.

Digital Library

[38]

Shewchuk, J. R. 1994. An introduction to the conjugate gradient method without the agonizing pain. Tech. rep., Carnegie Mellon University, School of Computer Science.

Digital Library

[39]

Shi, L., Yu, Y., Bell, N., and Feng, W.-W. 2006. A fast multigrid algorithm for mesh deformation. ACM Trans. Graph. 25, 3, 1108--1117.

Digital Library

[40]

Shi, X., Zhou, K., Tong, Y., Desbrun, M., Bao, H., and Guo, B. 2007. Mesh puppetry: Cascading optimization of mesh deformation with inverse kinematics. In ACM SIGGRAPH Papers. ACM, New York, 81.

Digital Library

[41]

Starck, J. and Hilton, A. 2005. Spherical matching for temporal correspondence of non-rigid surfaces. In Proceedings of the 10th IEEE International Conference on Computer Vision (ICCV'05). IEEE Computer Society, 1387--1394.

Digital Library

[42]

Sumner, R. W., Schmid, J., and Pauly, M. 2007. Embedded deformation for shape manipulation. In ACM SIGGRAPH Papers. ACM, New York, 80.

Digital Library

[43]

Sumner, R. W., Zwicker, M., Gotsman, C., and Popović, J. 2005. Mesh-Based inverse kinematics. ACM Trans. Graph. 24, 3, 488--495.

Digital Library

[44]

Süssmuth, J., Winter, M., and Greiner, G. 2008. Reconstructing animated meshes from time-varying point clouds. Comput. Graph. Forum 27, 5, 1469--1476.

Digital Library

[45]

Terzopoulos, D., Platt, J., Barr, A., and Fleischer, K. 1987. Elastically deformable models. In Proceedings of the 14th Annual Conference on Computer Graphics and Interactive Techniques. ACM, New York, 205--214.

Digital Library

[46]

Varanasi, K., Zaharescu, A., Boyer, E., and Horaud, R. P. 2008. Temporal surface tracking using mesh evolution. In Proceedings of the 10th European Conference on Computer Vision. Lecture Notes in Computer Science. Part II. Springer-Verlag, 30--43.

Digital Library

[47]

Wand, M., Jenke, P., Huang, Q., Bokeloh, M., Guibas, L., and Schilling, A. 2007. Reconstruction of deforming geometry from time-varying point clouds. In Proceedings of the 5th Eurographics Symposium on Geometry Processing (SGP'07). Eurographics Association, 49--58.

Digital Library

[48]

Weise, T., Leibe, B., and Gool, L. V. 2007. Fast 3D scanning with automatic motion compensation. In IEEE Conference on Computer Vision and Pattern Recognition (CVPR'07).

[49]

Würmlin, S., Lamboray, E., Staadt, O. G., and Gross, M. H. 2002. 3D video recorder. In Proceedings of the 10th Pacific Conference on Computer Graphics and Applications (PG '02) . IEEE Computer Society, 325.

Digital Library

[50]

Xu, W., Zhou, K., Yu, Y., Tan, Q., Peng, Q., and Guo, B. 2007. Gradient domain editing of deforming mesh sequences. In ACM SIGGRAPH Papers. ACM, New York, 84.

Digital Library

[51]

Zhang, L., Curless, B., and Seitz, S. M. 2003. Spacetime stereo: Shape recovery for dynamic scenes. In IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 367--374.

[52]

Zhang, L., Snavely, N., Curless, B., and Seitz, S. M. 2004. Spacetime faces: High-Resolution capture for modeling and animation. In ACM Annual Conference on Computer Graphics. 548--558.

Digital Library

[53]

Zhou, K., Huang, J., Snyder, J., Liu, X., Bao, H., Guo, B., and Shum, H.-Y. 2005. Large mesh deformation using the volumetric graph Laplacian. ACM Trans. Graph. 24, 3, 496--503.

Digital Library

[54]

Zitnick, C. L., Kang, S. B., Uyttendaele, M., Winder, S., and Szeliski, R. 2004. High-Quality video view interpolation using a layered representation. In ACM SIGGRAPH Papers. ACM, New York, 600--608.

Digital Library

[55]

Zwicker, M., Pfister, H., van Baar, J., and Gross, M. 2001. Surface splatting. In Proceedings of the 28th Annual Conference on Computer Graphics and Interactive Techniques. ACM, New York, 371--378.

Digital Library

Cited By

Chen RZhao JZhang FChalmers ARhee T(2024)Neural Radiance Fields for Dynamic View Synthesis Using Local Temporal PriorsComputational Visual Media10.1007/978-981-97-2095-8_5(74-90)Online publication date: 10-Apr-2024
https://dl.acm.org/doi/10.1007/978-981-97-2095-8_5
Leach ECowley EBowen C(2024)The experiences of podiatrists prescribing custom foot orthoses and patients using custom foot orthoses for foot pain management in the United Kingdom: A focus group studyJournal of Foot and Ankle Research10.1002/jfa2.1204717:3Online publication date: 21-Aug-2024
https://doi.org/10.1002/jfa2.12047
Yao YDeng BXu WZhang J(2023)Fast and Robust Non-Rigid Registration Using Accelerated Majorization-MinimizationIEEE Transactions on Pattern Analysis and Machine Intelligence10.1109/TPAMI.2023.324760345:8(9681-9698)Online publication date: Aug-2023
https://doi.org/10.1109/TPAMI.2023.3247603
Show More Cited By

Index Terms

Efficient reconstruction of nonrigid shape and motion from real-time 3D scanner data
1. Computing methodologies
  1. Artificial intelligence
    1. Computer vision
      1. Computer vision problems
      2. Computer vision tasks
        Scene understanding
  2. Computer graphics
    1. Animation
    2. Shape modeling
      1. Parametric curve and surface models

Recommendations

Real-time non-rigid reconstruction using an RGB-D camera

We present a combined hardware and software solution for markerless reconstruction of non-rigidly deforming physical objects with arbitrary shape in real-time. Our system uses a single self-contained stereo camera unit built from off-the-shelf ...
Robust single-view geometry and motion reconstruction

We present a framework and algorithms for robust geometry and motion reconstruction of complex deforming shapes. Our method makes use of a smooth template that provides a crude approximation of the scanned object and serves as a geometric and topological ...
3D surface reconstruction from multiview photographic images using 2D edge contours

Most techniques for reconstructing 3D shapes from multi-view 2D photographic images require a large number of images. In this paper, we present a new method for reconstructing 3D surfaces, represented by sets of polygons, using a small number, e.g. 10, ...

Comments

Information & Contributors

Information

Published In

cover image ACM Transactions on Graphics

ACM Transactions on Graphics Volume 28, Issue 2

April 2009

129 pages

ISSN:0730-0301

EISSN:1557-7368

DOI:10.1145/1516522

Issue’s Table of Contents

Copyright © 2009 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 13 May 2009

Accepted: 01 February 2009

Received: 01 November 2008

Published in TOG Volume 28, Issue 2

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed

Funding Sources

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

131
Total Citations
View Citations
2,262
Total Downloads

Downloads (Last 12 months)33
Downloads (Last 6 weeks)2

Reflects downloads up to 25 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Chen RZhao JZhang FChalmers ARhee T(2024)Neural Radiance Fields for Dynamic View Synthesis Using Local Temporal PriorsComputational Visual Media10.1007/978-981-97-2095-8_5(74-90)Online publication date: 10-Apr-2024
https://dl.acm.org/doi/10.1007/978-981-97-2095-8_5
Leach ECowley EBowen C(2024)The experiences of podiatrists prescribing custom foot orthoses and patients using custom foot orthoses for foot pain management in the United Kingdom: A focus group studyJournal of Foot and Ankle Research10.1002/jfa2.1204717:3Online publication date: 21-Aug-2024
https://doi.org/10.1002/jfa2.12047
Yao YDeng BXu WZhang J(2023)Fast and Robust Non-Rigid Registration Using Accelerated Majorization-MinimizationIEEE Transactions on Pattern Analysis and Machine Intelligence10.1109/TPAMI.2023.324760345:8(9681-9698)Online publication date: Aug-2023
https://doi.org/10.1109/TPAMI.2023.3247603
Xu WDu WXue HLi YYe RWang YLu C(2023)ClothPose: A Real-world Benchmark for Visual Analysis of Garment Pose via An Indirect Recording Solution2023 IEEE/CVF International Conference on Computer Vision (ICCV)10.1109/ICCV51070.2023.00012(58-68)Online publication date: 1-Oct-2023
https://doi.org/10.1109/ICCV51070.2023.00012
Joudah MKadhim FHasan S(2023)Scanner Method For Modeling and Manufacturing of 3d Printable Upper Limb Orthosis2023 Al-Sadiq International Conference on Communication and Information Technology (AICCIT)10.1109/AICCIT57614.2023.10218181(292-297)Online publication date: 4-Jul-2023
https://doi.org/10.1109/AICCIT57614.2023.10218181
Deng BYao YDyke RZhang J(2022)A Survey of Non‐Rigid 3D RegistrationComputer Graphics Forum10.1111/cgf.1450241:2(559-589)Online publication date: 24-May-2022
https://doi.org/10.1111/cgf.14502
Moschella LMelzi SCosmo LMaggioli FLitany OOvsjanikov MGuibas LRodolà E(2022)Learning Spectral Unions of Partial Deformable 3D ShapesComputer Graphics Forum10.1111/cgf.1448341:2(407-417)Online publication date: 24-May-2022
https://doi.org/10.1111/cgf.14483
Paranitharan KEbenezer GBalaji VAdham Khan MRamesh Babu T(2022)Application of industry 4.0 technology in containing Covid-19 spread and its challengesMaterials Today: Proceedings10.1016/j.matpr.2022.06.00968(1225-1232)Online publication date: 2022
https://doi.org/10.1016/j.matpr.2022.06.009
Fung SLu XMykolaitis MRazzak IKostkevičius GOzerenskis D(2022)Anatomical Landmarks Localization for 3D Foot Point CloudsNeural Information Processing10.1007/978-3-031-30111-7_53(627-638)Online publication date: 22-Nov-2022
https://dl.acm.org/doi/10.1007/978-3-031-30111-7_53
Farhan MWang JBray PBurns JCheng T(2021)Comparison of 3D scanning versus traditional methods of capturing foot and ankle morphology for the fabrication of orthoses: a systematic reviewJournal of Foot and Ankle Research10.1186/s13047-020-00442-814:1Online publication date: 7-Jan-2021
https://doi.org/10.1186/s13047-020-00442-8
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Issue’s Table of Contents