article

Global Non-rigid Alignment of Surface Sequences

Authors:

Martin Klaudiny,

Adrian HiltonAuthors Info & Claims

International Journal of Computer Vision, Volume 102, Issue 1-3

Pages 256 - 270

https://doi.org/10.1007/s11263-012-0553-4

Published: 01 March 2013 Publication History

Abstract

This paper presents a general approach based on the shape similarity tree for non-sequential alignment across databases of multiple unstructured mesh sequences from non-rigid surface capture. The optimal shape similarity tree for non-rigid alignment is defined as the minimum spanning tree in shape similarity space. Non-sequential alignment based on the shape similarity tree minimises the total non-rigid deformation required to register all frames in a database into a consistent mesh structure with surfaces in correspondence. This allows alignment across multiple sequences of different motions, reduces drift in sequential alignment and is robust to rapid non-rigid motion. Evaluation is performed on three benchmark databases of 3D mesh sequences with a variety of complex human and cloth motion. Comparison with sequential alignment demonstrates reduced errors due to drift and improved robustness to large non-rigid deformation, together with global alignment across multiple sequences which is not possible with previous sequential approaches.

References

[1]

Ahmed, N., Theobalt, C., Roessl, C., Thrun, S., & Seidel, H.-P. (2008). Dense correspondence finding for parameterization-free animation reconstruction from video. In Conference on computer vision and pattern recognition.

[2]

Baran, I., Vlasic, D., Grinspun, E., & Popovic, J. (2009). Semantic deformation transfer. In Proc. ACM SIGGRAPH.

[3]

Beeler, T., Hahn, F., Bradley, D., Bickel, B., Beardsley, P., Gotsman, C., & Gross, M. (2011). High-quality passive facial performance capture using anchor frames. In Proc. ACM SIGGRAPH.

[4]

Belongie, S., Malik, J., & Puzicha, J. (2002). Shape matching and object recognition using shape contexts. IEEE Transactions on Pattern Analysis and Machine Intelligence, 24(24), 509-522.

Digital Library

[5]

Botsch, M., & Sorkine, O. (2008). On linear variational surface deformation methods. IEEE Transactions on Visualization and Computer Graphics, 14(1), 213-230.

Digital Library

[6]

Bradley, D., Heidrich, W., Popa, T., & Sheffer, A. (2010). High-resolution passive facial performance capture. In Proc. ACM SIGGRAPH.

[7]

Bradley, D., Popa, T., Sheffer, A., Heidrich, W., & Boubekeur, T. (2008). Markerless garment capture. ACM Transactions on Graphics, 27(3), 99.

Digital Library

[8]

Bronstein, A., Bronstein, M., & Kimmel, R. (2007). Calculus of non-rigid surfaces for geometry and texture manipulation. IEEE Transactions on Visualization and Computer Graphics, 13(5), 902-913.

Digital Library

[9]

Budd, C., & Hilton, A. (2009). Skeleton driven volumetric Laplacian deformation. In European conference on visual media production.

[10]

Budd, C., Huang, P., & Hilton, A. (2011). Hierarchical shape matching for temporally consistent 3D video. In IEEE conf. 3D imaging, processing, visualisation and transmission (3DIMPVT).

[11]

Cagniart, C., Boyer, E., & Ilic, S. (2010a). Free-form mesh tracking: a patch-based approach. In Conference on computer vision and pattern recognition.

[12]

Cagniart, C., Boyer, E., & Ilic, S. (2010b). Probabilistic deformable surface tracking from multiple videos. In European conference on computer vision.

[13]

Carceroni, R., & Kutulakos, K. (2002). Multi-view scene capture by surfel sampling: from video streams to non-rigid 3D motion, shape and reflectance. International Journal of Computer Vision, 49(2-3), 175-214.

[14]

Carranza, J., Theobalt, C., Magnor, M., & Seidel, H.-P. (2003). Free-viewpoint video of human actors. In Proceedings ACM SIGGRAPH (Vol. 22(3), pp. 569-577).

[15]

de Aguiar, E., Stoll, C., Theobalt, C., Ahmed, N., Seidel, H.-P., & Thrun, S. (2008). Performance capture from sparse multiview video. In Proceedings of the ACM SIGGRAPH (Vol. 27(3)).

[16]

Elad, A., & Kimmel, R. (2003). On bending invariant signatures for surfaces. IEEE Transactions on Pattern Analysis and Machine Intelligence, 25(10), 1285-1295.

Digital Library

[17]

Enqvist, O., Kahl, F., & Olsson, C. (2011). Non-sequential structure from motion. In IEEE workshop on omnidirectional vision, camera networks and non-classical camera.

[18]

Furukawa, Y., & Ponce, J. (2010). Dense 3D motion capture for human faces. In Conference on computer vision and pattern recognition.

[19]

Gal, R., Shamir, A., & Cohen-Or, D. (2007). Pose oblivious shape signature. IEEE Transactions on Visualization and Computer Graphics, 13(2), 261-271.

Digital Library

[20]

Gatzke, T., Grimm, C., Garland, M., & Zelinka, S. (2005). Curvature maps for local shape comparison. In Shape modelling and applications.

[21]

Gherhadi, R., Toldo, R., Farenzena, M., & Fusiello, A. (2010). Samantha: towards automatic image-based model acquisition. In Proceedings of the 7th European conference on visual media production (CVMP 2010).

[22]

Huang, P., Budd, C., & Hilton, A. (2011). Global temporal registration of multiple non-rigid surface sequences. In Conference on computer vision and pattern recognition.

[23]

Huang, P., Hilton, A., & Starck, J. (2009). Human motion synthesis from 3D video. In IEEE int. conf. on computer vision and pattern recognition, CVPR.

[24]

Huang, P., Hilton, A., & Starck, J. (2010). Shape similarity for 3D video sequences of people. International Journal of Computer Vision, 89(2-3), 362-381.

Digital Library

[25]

Iyer, N., Jayanti, S., Lou, K., Kalyanaraman, Y., & Ramani, K. (2005). Three dimensional shape searching: state-of-the-art review and future trends. Computer Aided Design, 37(5).

[26]

Kruskal, J. B. (1956). On the shortest spanning subtree of a graph and the traveling salesman problem. Proceedings of the American Mathematical Society, 7(1), 48-50.

[27]

Liing, H., & Jacobs, D. (2005). Deformation invariant image matching. In IEEE int. conf. on computer vision (pp. 1466-1473).

[28]

Lowe, D. (2004). Distinctive image features for scale invariant key-points. International Journal of Computer Vision, 60(2), 91-110.

[29]

McInerney, T., & Terzopoulos, D. (1996). Deformable models in medical image processing. Medical Image Analysis, 1(2), 91-108.

[30]

Mikolajczyk, K., & Schmid, C. (2005). A performance evaluation of local descriptors. IEEE Transactions on Pattern Analysis and Machine Intelligence, 27(10), 1615-1630.

Digital Library

[31]

Moeslund, T., Hilton, A., & Kruger, V. (2006). A survey of advances in vision-based human motion capture and analysis. Computer Vision and Image Understanding, 104(2-3), 90-127.

Digital Library

[32]

Moeslund, T., Hilton, A., Kruger, V., & Sigal, L. E. (2011). Visual analysis of humans: looking at people. Berlin: Springer.

[33]

Neumann, J., & Aloimonos, Y. (2002). Spatio-temporal stereo using multi-resolution subdivision surfaces. International Journal of Computer Vision, 47(1-3), 181-193.

[34]

Pons, J.-P., Keriven, R., & Faugeras, O. (2007). Multi-view stereo reconstruction and scene flow estimation with a global image-based matching score. International Journal of Computer Vision, 72(2), 179-193.

Digital Library

[35]

Prim, R. C. (1957). Shortest connection networks and some generalizations. Bell System Technology Journal, 36, 1389-1401.

[36]

Pritchard, D., & Heidrich, W. (2003). Cloth motion capture. Computer Graphics Forum, 22(3), 263-271.

[37]

Salzmann, M., Pilet, J., Ilic, S., & Fua, P. (2007). Surface deformation models for non-rigid 3D shape recovery. In IEEE trans. Pattern analysis and machine intelligence.

[38]

Scholz, V., Stich, T., Kechkeisen, M., Wacker, M., & Magnor, M. (2005). Garment motion capture using color-coded patterns. Computer Graphics Forum, 24(3), 439-448.

[39]

Sorkine, O. (2006). Differential representations for mesh processing. Computer Graphics Forum, 25(4).

[40]

Starck, J., & Hilton, A. (2003). Model-based multiple view reconstruction of people. In IEEE international conference on computer vision (pp. 915-922).

[41]

Starck, J., & Hilton, A. (2005). Spherical matching for temporal correspondence of non-rigid surfaces. In IEEE int. conf. computer vision (pp. 1387-1394).

[42]

Starck, J., & Hilton, A. (2007a). Correspondence labelling for wide-timeframe free-form surface matching. In IEEE int. conf. on computer vision.

[43]

Starck, J., & Hilton, A. (2007b). Surface capture for performance-based animation. IEEE Computer Graphics and Applications, 27(3), 21-31.

Digital Library

[44]

Stoll, C., Gall, J., de Aguiar, E., Thrun, S., & Theobalt, C. (2010). Video-based reconstruction of animatable human characters. In ACM SIGGRAPH ASIA.

[45]

Sumner, R., & Popovic, J. (2004). Deformation transfer for triangle meshes. In Proc. ACM SIGGRAPH.

[46]

Tevs, A., Berner, A., Wand, M., Ihrke, I., Bokeloh, M., Kerber, J., & Seidel, H.-P. (2011). Animation cartography--intrinsic reconstruction of shape and motion. ACM Transaction on Graphics.

[47]

Tung, T., & Matsuyama, T. (2010). Dynamic surface matching by geodesic mapping for animation transfer. In Conference on computer vision and pattern recognition.

[48]

Vedula, S., Baker, S., Rander, P., Collins, R., & Kanade, T. (2005). Three-dimensional scene flow. IEEE Transactions Pattern Analysis and Machine Intelligence, 27(3).

[49]

Vlasic, D., Baran, I., Matusik, W., & Popovic, J. (2008). Articulated mesh animation from multi-view silhouettes. In Proc. ACM SIGGRAPH.

[50]

Wand, M., Adams, B., Ovsianikov, M., Berner, A., Bokeloh, M., Jenke, P., Guibas, L., Seidel, H.-P., & Schilling, A. (2009). Efficient reconstruction of non-rigid shape and motion from real-time 3D scanner data. ACM Transaction on Graphics, 28(2).

[51]

White, R., Crane, K., & Forsythe, D. (2007). Capturing and animating occluded cloth. In Proc. ACM SIGGRAPH (Vol. 26(3), p. 34).

[52]

Zeng, Y., Wang, C., Wang, Y., Gu, X., Samaras, D., & Paragios, N. (2010). Dense non-rigid surface registration using high-order graph matching. In Conference on computer vision and pattern recognition.

[53]

Zhang, L., Snavely, N., Curless, B., & Seitz, S. (2004). Spacetime faces: high resolution capture for modelling and animation. In Proc. ACM SIGGRAPH (pp. 546-556).

Cited By

Dvořák JHácha FVáša L(2023)Global Optimisation for Improved Volume Tracking of Time-Varying MeshesComputational Science – ICCS 202310.1007/978-3-031-36027-5_9(113-127)Online publication date: 3-Jul-2023
https://dl.acm.org/doi/10.1007/978-3-031-36027-5_9
MacQuarrie ASteed A(2020)Exploring the Use of Skeletal Tracking for Cheaper Motion Graphs and On-Set Decision Making in Free-Viewpoint Video ProductionProceedings of the 17th ACM SIGGRAPH European Conference on Visual Media Production10.1145/3429341.3429353(1-10)Online publication date: 7-Dec-2020
https://dl.acm.org/doi/10.1145/3429341.3429353
Morgenstern WHilsmann AEisert P(2019)Progressive Non-rigid Registration of Temporal Mesh SequencesProceedings of the 16th ACM SIGGRAPH European Conference on Visual Media Production10.1145/3359998.3369411(1-10)Online publication date: 17-Dec-2019
https://dl.acm.org/doi/10.1145/3359998.3369411
Show More Cited By

Global Non-rigid Alignment of Surface Sequences
1. Computing methodologies
  1. Artificial intelligence
    1. Computer vision
      1. Computer vision tasks
      2. Image and video acquisition
  2. Computer graphics
2. Theory of computation

Recommendations

Towards optimal non-rigid surface tracking
ECCV'12: Proceedings of the 12th European conference on Computer Vision - Volume Part IV

This paper addresses the problem of optimal alignment of non-rigid surfaces from multi-view video observations to obtain a temporally consistent representation. Conventional non-rigid surface tracking performs frame-to-frame alignment which is subject ...
Global temporal registration of multiple non-rigid surface sequences
CVPR '11: Proceedings of the 2011 IEEE Conference on Computer Vision and Pattern Recognition

In this paper we consider the problem of aligning multiple non-rigid surface mesh sequences into a single temporally consistent representation of the shape and motion. A global alignment graph structure is introduced which uses shape similarity to ...
Cooperative patch-based 3D surface tracking
CVMP '11: Proceedings of the 2011 Conference for Visual Media Production

This paper presents a novel dense motion capture technique which creates a temporally consistent mesh sequence from several calibrated and synchronised video sequences of a dynamic object. A surface patch model based on the topology of a user-specified ...

Comments

Information & Contributors

Information

Published In

cover image International Journal of Computer Vision

International Journal of Computer Vision Volume 102, Issue 1-3

March 2013

307 pages

ISSN:0920-5691

Issue’s Table of Contents

Copyright © Copyright © 2013 Springer Science+Business Media New York.

Publisher

Kluwer Academic Publishers

United States

Publication History

Published: 01 March 2013

Author Tags

Qualifiers

Article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

21
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 15 Oct 2024

Other Metrics

View Author Metrics

Citations

Cited By

Dvořák JHácha FVáša L(2023)Global Optimisation for Improved Volume Tracking of Time-Varying MeshesComputational Science – ICCS 202310.1007/978-3-031-36027-5_9(113-127)Online publication date: 3-Jul-2023
https://dl.acm.org/doi/10.1007/978-3-031-36027-5_9
MacQuarrie ASteed A(2020)Exploring the Use of Skeletal Tracking for Cheaper Motion Graphs and On-Set Decision Making in Free-Viewpoint Video ProductionProceedings of the 17th ACM SIGGRAPH European Conference on Visual Media Production10.1145/3429341.3429353(1-10)Online publication date: 7-Dec-2020
https://dl.acm.org/doi/10.1145/3429341.3429353
Morgenstern WHilsmann AEisert P(2019)Progressive Non-rigid Registration of Temporal Mesh SequencesProceedings of the 16th ACM SIGGRAPH European Conference on Visual Media Production10.1145/3359998.3369411(1-10)Online publication date: 17-Dec-2019
https://dl.acm.org/doi/10.1145/3359998.3369411
Casas DOtaduy M(2018)Learning Nonlinear Soft-Tissue Dynamics for Interactive AvatarsProceedings of the ACM on Computer Graphics and Interactive Techniques10.1145/32031871:1(1-15)Online publication date: 25-Jul-2018
https://dl.acm.org/doi/10.1145/3203187
Gilbert AVolino MCollomosse JHilton A(2018)Volumetric Performance Capture from Minimal Camera ViewpointsComputer Vision – ECCV 201810.1007/978-3-030-01252-6_35(591-607)Online publication date: 8-Sep-2018
https://dl.acm.org/doi/10.1007/978-3-030-01252-6_35
Prada FKazhdan MChuang MCollet AHoppe H(2017)Spatiotemporal atlas parameterization for evolving meshesACM Transactions on Graphics10.1145/3072959.307367936:4(1-12)Online publication date: 20-Jul-2017
https://dl.acm.org/doi/10.1145/3072959.3073679
Robertini NCasas DAguiar ETheobalt C(2017)Multi-view Performance Capture of Surface DetailsInternational Journal of Computer Vision10.1007/s11263-016-0979-1124:1(96-113)Online publication date: 1-Aug-2017
https://dl.acm.org/doi/10.1007/s11263-016-0979-1
Fechteler PHilsmann AEisert PSantos LBashford-Rogers T(2016)Example-based body model optimization and skinningProceedings of the 37th Annual Conference of the European Association for Computer Graphics: Short Papers10.5555/3059107.3059110(5-8)Online publication date: 9-May-2016
https://dl.acm.org/doi/10.5555/3059107.3059110
Casas DFeng AAlexander OFyffe GDebevec PIchikari RLi HOlszewski KSuma EShapiro AThalmann NGavrilova MKomura TCordier FThalmann D(2016)Rapid Photorealistic Blendshape Modeling from RGB-D SensorsProceedings of the 29th International Conference on Computer Animation and Social Agents10.1145/2915926.2915936(121-129)Online publication date: 23-May-2016
https://dl.acm.org/doi/10.1145/2915926.2915936
Biasotti SCerri ABronstein ABronstein M(2016)Recent Trends, Applications, and Perspectives in 3D Shape Similarity AssessmentComputer Graphics Forum10.1111/cgf.1273435:6(87-119)Online publication date: 1-Sep-2016
https://dl.acm.org/doi/10.1111/cgf.12734
Show More Cited By

View Options

View options

Get Access

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

Media

Figures

Other

Tables

View Issue’s Table of Contents