Non-Rigid Structure-From-Motion on Degenerate Deformations With Low-Rank Shape Deformation Model
Authors: 
Zhong Zhou, Feng Shi, Jiangjian Xiao, Wei WuAuthors Info & Claims
Zhong Zhou, Feng Shi, Jiangjian Xiao, Wei WuAuthors Info & ClaimsPages 171 - 185
Abstract
Non-rigid structure-from-motion (NRSfM) is the process of recovering time-varying 3D structures and poses of a deformable object from an uncalibrated monocular video sequence. Currently, most NRSfM algorithms utilize a non- degenerate assumption for non-rigid object deformations whereby the 3D structures of a non-rigid object can be assumed to be a linear combination of basis shapes with full rank three. Unfortunately, this assumption will produce extra degrees-of-freedom when the non-rigid object has some degenerate deformations with shape bases of rank less than three. These extra degrees-of-freedom will yield spurious shape deformations due to non-negligible noise in real applications, which will cause substantial reconstruction errors. To solve this problem, we propose a low-rank shape deformation model to represent 3D structures of degenerate deformations. When modeling degenerate deformations, the proposed model exploits the rank-deficient nature of degenerate deformations in addition to the low-rank property of non-rigid objects' trajectories, thus providing a more accurate and compact representation compared with existing models. Based on this model, we formulate the NRSfM problem as two coherent optimization problems. These problems are solved with iterative non-linear optimization algorithms. Experiments on synthetic and motion capture data are conducted. The results exhibit the significant advantages of our approach over state-of-the-art NRSfM algorithms for the 3D recovery of non-rigid objects with degenerate deformations.
References
[1]
C. Tomasi and T. Kanade, “Shape and motion from image streams under orthography: A factorization method,” Int. J. Comput. Vis., vol. 9, pp. 137–154, 1992.
[2]
C. Bregler, A. Hertzmann, and H. Biermann, “Recovering non-rigid 3D shape from image streams,” in Proc. IEEE Conf. Comput. Vis. Pattern Recog., Jun. 2000, vol. 2, pp. 690–696.
[3]
J. Xiao and T. Kanade, “A closed-form solution to non-rigid shape and motion recovery,” in Proc. Eur. Conf. Comput. Vis., 2004, pp. 196–209.
[4]
M. Brand, “A direct method for 3D factorization of nonrigid motion observed in 2D,” in Proc. IEEE Conf. Comput. Vis. Pattern Recog., Jun. 2005, vol. 2, pp. 122–128.
[5]
I. Akhter, Y. Sheikh, and S. Khan, “In defense of orthonormality constraints for nonrigid structure from motion,” in Proc. IEEE Conf. Comput. Vis. Pattern Recog., Jun. 2009, pp. 1534–1541.
[6]
L. Torresani, A. Hertzmann, and C. Bregler, “Nonrigid structure-from-motion: Estimating shape and motion with hierarchical priors,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 30, no. 5, pp. 878–892, May 2008.
[7]
S. Olsen and A. Bartoli, “Implicit non-rigid structure-from-motion with priors,” J. Math. Imag. Vis., vol. 31, no. 2–3, pp. 233–244, 2008.
[8]
M. Paladini, A. D. Bue, M. Stosic, M. Dodig, J. Xavier, and L. Agapito, “Factorization for non-rigid and articulated structure using metric projections,” in Proc. IEEE Conf. Comput. Vis. Pattern Recog., Jun. 2009, pp. 2898–2905.
[9]
V. Rabaud and S. Belongie, “Linear embeddings in non-rigid structure from motion,” in Proc. IEEE Conf. Comput. Vis. Pattern Recog., Jun. 2009, pp. 2427–2434.
[10]
Y. Dai, H. Li, and M. He, “A simple prior-free method for non-rigid structure-from-motion factorization,” in Proc. IEEE Conf. Comput. Vis. Pattern Recog., Jun. 2012, pp. 2018–2025.
[11]
H. Zhou, X. Li, and A. H. Sadka, “Nonrigid structure-from-motion from 2-D images using Markov chain Monte Carlo,” IEEE Trans. Multimedia, vol. 14, no. 1, pp. 168–177, Jun. 2012.
[12]
I. Akhter, Y. Sheikh, S. Khan, and T. Kanade, “Nonrigid structure from motion in trajectory space,” in Proc. Neural Inf. Process. Syst., 2008, pp. 41–48.
[13]
I. Akhter, Y. Sheikh, S. Khan, and T. Kanade, “Trajectory space: A dual representation for nonrigid structure from motion,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 33, no. 7, pp. 1442–1456, Jul. 2011.
[14]
P. F. U. Gotardo and A. M. Martinez, “Non-rigid structure from motion with complementary rank-3 spaces,” in Proc. IEEE Conf. Comput. Vis. Pattern Recog., Jun. 2011, pp. 3065–3072.
[15]
P. F. U. Gotardo and A. M. Martinez, “Kernel non-rigid structure from motion,” in Proc. IEEE Int. Conf. Comput. Vis., Nov. 2011, pp. 802–809.
[16]
I. Khan, “Non-rigid structure-from-motion with uniqueness constraint and low rank matrix fitting factorization,” IEEE Trans. Multimedia, vol. 16, no. 5, pp. 1350–1357, Aug. 2014.
[17]
J. Xiao and T. Kanade, “Non-rigid shape and motion recovery: Degenerate deformations,” in Proc. IEEE Conf. Comput. Vis. Pattern Recog., Jun.–Jul. 2004, vol. 1, pp. 668–675.
[18]
M. Paladini, A. Bartoli, and L. Agapito, “Sequential non-rigid structure-from-motion with the 3D-implicit low-rank shape model,” in Proc. Eur. Conf. Comput. Vis., 2010, pp. 15–28.
[19]
R. Angst and M. Pollefeys, “A unified view on deformable shape factorizations,” in Proc. Eur. Conf. Comput. Vis., 2012, pp. 682–695.
[20]
P. F. U. Gotardo and A. M. Martinez, “Computing smooth time-trajectories for camera and deformable shape in structure from motion with occlusion,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 33, no. 10, pp. 2051–2065, Oct. 2011.
[21]
W. H. Press, S. A. Teukolsky, W. T. Vetterling, and B. P. Flannery, Numerical Recipes in C, 2nd ed., Cambridge, U.K.: Cambridge Univ. Press, 1992.
[22]
J. R. Magnus and H. Neudecker, Matrix Differential Calculus with Applications in Statistics and Econometrics, 2nd ed., New York, NY USA: Wiley, 1999.
[23]
I. Matthews and S. Baker, “Active appearance models revisited,” Int. J. Comput. Vision, vol. 60, no. 2, pp. 135–164, 2004.
[24]
R. Tron and R. Vidal, “A benchmark for the comparison of 3D motion segmentation algorithms,” in Proc. IEEE Conf. Comput. Vis. Pattern Recog., Jun. 2007, pp. 1–8.
[25]
L. Ding and A. M. Martinez, “Modelling and recognition of the linguistic components in american sign language,” Image Vis. Comput., vol. 27, no. 12, pp. 1826–1844, 2009.
[26]
F. Rengier, A. Mehndiratta, H. von Tengg-Kobligk, C. M. Zechmann, R. Unterhinninghofen, H.-U. Kauczor, and F. L. Giesel, “3D printing based on imaging data: Review of medical applications,” Int. J. Comput. Assisted Radiol. Surgery, vol. 5, no. 4, pp. 335–341, 2010.
[27]
J. Zhu, S. C. Hoi, and M. R. Lyu, “Real-time non-rigid shape recovery via active appearance models for augmented reality,” in Proc. Eur. Conf. Comput. Vis., 2006, pp. 186–197.
[28]
J. Sivic and A. Zisserman, “Video Google: A text retrieval approach to object matching in videos,” in Proc. IEEE Int. Conf. Comput. Vis., Oct. 2003, vol. 2, pp. 1470–1477.
[29]
P. Ochs, J. Malik, and T. Brox, “Segmentation of moving objects by long term video analysis,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 36, no. 6, pp. 1187–1200, Jun. 2014.
[30]
H. S. Park, T. Shiratori, I. Matthews, and Y. Sheikh, “3D reconstruction of a moving point from a series of 2D projections,” in Proc. Eur. Conf. Comput. Vis., 2010, pp. 158–171.
[31]
A. Zaheer, I. Akhter, M. H. Baig, S. Marzban, and S. Khan, “Multiview structure from motion in trajectory space,” in Proc. IEEE Int. Conf. Comput. Vis., Nov. 2011, pp. 2447–2453.
[32]
A. D. Bue, “Adaptive non-rigid registration and structure from motion from image trajectories,” Int. J. Comput. Vis., vol. 103, no. 2, pp. 226–239, 2013.
[33]
L. Tao and B. J. Matuszewski, “Non-rigid structure from motion with diffusion maps prior,” in Proc. IEEE Conf. Comput. Vis. Pattern Recog., Jun. 2013, pp. 1530–1537.
[34]
H. Liang, J. Yuan, and D. Thalmann, “Parsing the hand in depth images,” IEEE Trans. Multimedia, vol. 16, no. 5, pp. 1241–1253, Aug. 2014.
[35]
A. Agudo, L. Agapito, B. Calvo, and J. M. M. Montiel, “Good vibrations: A modal analysis approach for sequential non-rigid structure from motion,” in Proc. IEEE Conf. Comput. Vis. Pattern Recog., Jun. 2014, pp. 1558–1565.
Index Terms
- Non-Rigid Structure-From-Motion on Degenerate Deformations With Low-Rank Shape Deformation Model
Index terms have been assigned to the content through auto-classification.
Recommendations
2D shape deformation based on rigid square matching
CASA'2008 Special IssueIn this paper, we propose a fast and stable method for 2D shape deformation based on rigid square matching. Our method utilizes uniform quadrangular control meshes for 2D shapes and tries to maintain the rigidity of each square in the control mesh ...
Medial-axis-driven shape deformation with volume preservation
The medial axis is a natural skeleton for shapes. However, it is rarely used in the existing skeleton-based shape deformation techniques. In this paper, we propose a novel medial-axis-driven skin surface deformation algorithm with volume preservation ...
Estimation of Non-rigid Surface Deformation Using Developable Surface Model
ICPR '10: Proceedings of the 2010 20th International Conference on Pattern RecognitionThere is a strong demand for a method of acquiring a non-rigid shape under deformation with high accuracy and high resolution. However, this is difficult to achieve because of performance limitations in measurement hardware. In this paper, we propose a ...
Comments
Information & Contributors
Information
Published In
Copyright © 2014.
Publisher
IEEE Press
Publication History
Published: 01 February 2015
Qualifiers
- Research-article
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 0Total Downloads
- Downloads (Last 12 months)0
- Downloads (Last 6 weeks)0
Reflects downloads up to 10 Feb 2025