article

Robust Algebraic Segmentation of Mixed Rigid-Body and Planar Motions from Two Views

Authors:

Shankar R. Rao,

S. Shankar Sastry,

Yi MaAuthors Info & Claims

International Journal of Computer Vision, Volume 88, Issue 3

Pages 425 - 446

https://doi.org/10.1007/s11263-009-0314-1

Published: 01 July 2010 Publication History

Abstract

This paper studies segmentation of multiple rigid-body motions in a 3-D dynamic scene under perspective camera projection. We consider dynamic scenes that contain both 3-D rigid-body structures and 2-D planar structures. Based on the well-known epipolar and homography constraints between two views, we propose a hybrid perspective constraint (HPC) to unify the representation of rigid-body and planar motions. Given a mixture of K hybrid perspective constraints, we propose an algebraic process to partition image correspondences to the individual 3-D motions, called Robust Algebraic Segmentation (RAS). Particularly, we prove that the joint distribution of image correspondences is uniquely determined by a set of (2 K )-th degree polynomials, a global signature for the union of K motions of possibly mixed type. The first and second derivatives of these polynomials provide a means to recover the association of the individual image samples to their respective motions. Finally, using robust statistics, we show that the polynomials can be robustly estimated in the presence of moderate image noise and outliers. We conduct extensive simulations and real experiments to validate the performance of the new algorithm. The results demonstrate that RAS achieves notably higher accuracy than most existing robust motion-segmentation methods, including random sample consensus (RANSAC) and its variations. The implementation of the algorithm is also two to three times faster than the existing methods. The implementation of the algorithm and the benchmark scripts are available at http://perception.csl.illinois.edu/ras/ .

References

[1]

Agrawal, R., Gehrke, J., Gunopulos, D., & Raghavan, P. (1998). Automatic subspace clustering of high dimensional data for data mining applications. In Proceedings of ACM special interest group on management of data .

[2]

Anandan, P., & Avidan, S. (2000). Integrating local affine into global projective images in the joint image space. In Proceedings of European conference on computer vision .

[3]

Ballard, D. (1981). Generalizing the Hough transform to detect arbitrary patterns. Pattern Recognition, 13 (2), 111-122.

[4]

Bartoli, A. (2001). Piecewise planar segmentation for automatic scene modeling. In Proceedings of IEEE conference on computer vision and pattern recognition .

[5]

Campbell, N. (1978). The influence function as an aid in outlier detection in discriminant analysis. Applied Statistics, 27 (3), 251-258.

[6]

Campbell, N. (1980). Robust procedures in multivariate analysis I: robust covariance estimation. Applied Statistics, 29 (3), 231-237.

[7]

Costeira, J., & Kanade, T. (1998). A multibody factorization method of independently moving objects. International Journal on Computer Vision, 29 (3), 159-179.

[8]

Fischler, M., & Bolles, R. (1981). Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography. Communications of the ACM, 24 , 381- 85.

Digital Library

[9]

Frahm, J., & Pollefeys, M. (2006). RANSAC for (quasi-) degenerate data (QDEGSAC). In Proceedings of IEEE conference on computer vision and pattern recognition (pp. 453-460).

[10]

Gnanadesikan, R., & Kettenring, J. (1972). Robust estimates, residuals, and outlier detection with multiresponse data. Biometrics, 28 (1), 81-124.

[11]

Goh, A., & Vidal, R. (2007). Segmenting motions of different types by unsupervised manifold clustering. In Proceedings of IEEE conference on computer vision and pattern recognition (pp. 1-6).

[12]

Goshen, L., Shimshoni, I., Anandan, P.,& Keren, D. (2005). Motion recovery by integrating over the joint image manifold. International Journal on Computer Vision, 65 (3), 131-145.

Digital Library

[13]

Hampel, F., Ronchetti, E., Rousseeuw, P., & Stahel, W. (1986). Robust statistics: the approach based on influence functions . New York: Wiley.

[14]

Han, M., & Kanade, T. (2000). Reconstruction of a scene with multiple linearly moving objects. In Proceedings of IEEE conference on computer vision and pattern recognition (Vol. 2, pp. 542-549).

[15]

Harris, J. (1992). Algebraic geometry: a first course . Berlin: Springer.

[16]

Huber, P. (1981). Robust statistics . New York: Wiley.

[17]

Jollife, I. (2002). Principal component analysis (2nd edn.). Berlin: Springer.

[18]

Kanatani, K. (2002a). Estimating the number of independent motions for multibody segmentation. In Proceedings of Asian conference on computer vision (pp. 7-12).

[19]

Kanatani, K. (2002b). Motion segmentation by subspace separation: model selection and reliability evaluation. International Journal of Image and Graphics, 2 (2), 179-197.

[20]

Kanatani, K., & Sugaya, Y. (2003). Multi-stage optimization for multibody motion segmentation. In Proceedings of the Australia-Japan advanced workshop on computer vision .

[21]

Lang, S. (2002). Algebra . Berlin: Springer.

[22]

Lee, K., Ho, J., & Kriegman, D. (2005). Acquiring linear subspaces for face recognition under variable lighting. IEEE Transactions on Pattern Analysis and Machine Intelligence, 27 (5), 684-698.

Digital Library

[23]

Leonardis, A., Bischof, H., & Maver, J. (2002). Multiple eigenspaces. Pattern Recognition, 35 (11), 2613-2627.

[24]

Ma, Y., Derksen, H., Hong, W., & Wright, J. (2007). Segmentation of multivariate mixed data via lossy coding and compression. IEEE Transactions on Pattern Analysis and Machine Intelligence, 29 (9), 1546-1562.

Digital Library

[25]

Ma, Y., Yang, A., Derksen, H., & Fossum, R. (2008). Estimation of subspace arrangements with applications in modeling and segmenting mixed data. SIAM Review, 50 (3), 413-458.

Digital Library

[26]

Ng, A., Weiss, Y., & Jordan, M. (2001). On spectral clustering: analysis and an algorithm. In Proceedings of neural and information processing systems .

[27]

Ozden, K., Schindler, K., & van Gool, L. (2007). Simultaneous segmentation and reconstruction of monocular image sequences. In Proceedings of IEEE international conference on computer vision .

[28]

Rao, S., Yang, A., Wagner, A., & Ma, Y. (2005). Segmentation of hybrid motions via hybrid quadratic surface analysis. In Proceedings of IEEE international conference on computer vision .

[29]

Rao, S., Tron, R., Vidal, R., & Ma, Y. (2008). Motion segmentation via robust subspace separation in the presence of outlying, incomplete, or corrupted trajectories. In Proceedings of IEEE conference on computer vision and pattern recognition .

[30]

Rousseeuw, P. (1984). Least median of squares regression. Journal of American Statistics Association, 79 , 871-880.

[31]

Schindler, K., & Suter, D. (2005). Two-view multibody structure-and-motion with outliers. In Proceedings of IEEE conference on computer vision and pattern recognition .

[32]

Scholkopf, B., Smola, A., & Muller, K. (1998). Nonlinear component analysis as a kernel eigenvalue problem. Neural Computation, 10 , 1299-1319.

Digital Library

[33]

Shapiro, L. (1995). Affine analysis of image sequences . Cambridge: Cambridge University Press.

[34]

Shashua, A., & Levin, A. (2001). Multi-frame infinitesimal motion model for the reconstruction of (dynamic) scenes with multiple linearly moving objects. In Proceedings of IEEE international conference on computer vision (Vol. 2, pp. 592-599).

[35]

Souvenir, R., & Pless, R. (2007).Manifold Clustering. In International conference on computer vision .

[36]

Stewart, C. (1995). MINPRAN: a new robust estimator for computer vision. IEEE Transactions on Pattern Analysis and Machine Intelligence, 17 (10), 925-938.

Digital Library

[37]

Stewart, C. (1999). Robust parameter estimation in computer vision. SIAM Review, 41(3), 513-537.

Digital Library

[38]

Subbarao, R., & Meer, P. (2006). Beyond RANSAC: user independent robust regression. In CVPR workshop on 25 years of RANSAC .

[39]

Sugaya, Y., & Kanatani, K. (2003). Outlier removal for motion tracking by subspace separation. IEICE Transactions on Information and Systems, E86-D (6), 1095-1102.

[40]

Taubin, G. (1991). Estimation of planar curves, surfaces, and nonplanar space curves defined by implicit equations with applications to edge and range image segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 13 (11), 1115-1138.

Digital Library

[41]

Tipping, M., & Bishop, C. (1999). Mixtures of probabilistic principal component analyzers. Neural Computation, 11 , 443-482.

Digital Library

[42]

Tong, W., Tang, C., & Medioni, G. (2004). Simultaneous two-view epipolar geometry estimation and motion segmentation by 4D tensor voting. IEEE Transactions on Pattern Analysis and Machine Intelligence, 26 (9), 1167-1184.

Digital Library

[43]

Torr, P. (1998). Geometric motion segmentation and model selection. Philosophical Transactions of the Royal Society of London, 356 (1740), 1321-1340.

[44]

Torr, P., & Davidson, C. (2003). IMPSAC: synthesis of importance sampling and random sample consensus. IEEE Transactions on Pattern Analysis and Machine Intelligence, 25 (3), 354-364.

Digital Library

[45]

Torr, P., & Zisserman, A. (2000). MLESAC: A new robust estimator with application to estimating image geometry. Computer Vision and Image Understanding , (78), 138-156.

[46]

Triggs, B. (1995). Matching constraints and the joint image. In Proceedings of IEEE international conference on computer vision .

[47]

Tron, R., & Vidal, R. (2007). A benchmark for the comparison of 3-D motion segmentation algorithms. In Proceedings of IEEE conference on computer vision and pattern recognition (pp. 1-8).

[48]

Vidal, R., & Hartley, R. (2008). Three-view multibody structure from motion. IEEE Transactions on Pattern Analysis and Machine Intelligence, 30 (2), 214-227.

Digital Library

[49]

Vidal, R., & Ma, Y. (2006). A unified algebraic approach to 2-D and 3-D motion segmentation and estimation. Journal of Mathematical Imaging and Vision, 25 , 403-421.

Digital Library

[50]

Vidal, R., Ma, Y., & Sastry, S. (2005). Generalized principal component analysis (GPCA). IEEE Transactions on Pattern Analysis and Machine Intelligence, 27 (12), 1-15.

Digital Library

[51]

Vidal, R., Ma, Y., Soatto, S., & Sastry, S. (2006). Two-view multibody structure from motion. International Journal on Computer Vision, 68 (1), 7-25.

Digital Library

[52]

Wang, H., & Suter, D. (2004). MDPE: a very robust estimator for model fitting and range image segmentation. International Journal on Computer Vision, 59 (2), 139-166.

[53]

Weiss, Y. (1999). Segmentation using eigenvectors: a unifying view. In Proceedings of IEEE international conference on computer vision (pp. 975-982).

[54]

Yan, J., & Pollefeys, M. (2006). A general framework for motion segmentation: independent, articulated, rigid, non-rigid, degenerate and non-degenerate. In Proceedings of European conference on computer vision (pp. 94-106).

[55]

Yang, A. (2006). Estimation of subspace arrangements: its algebra and statistics . PhD Dissertation, University of Illinois at Urbana-Champaign.

[56]

Yang, A., Rao, S., Wagner, A., & Ma, Y. (2005). Segmentation of a piece-wise planar scene from perspective images. In Proceedings of IEEE conference on computer vision and pattern recognition .

[57]

Yang, A., Rao, S., & Ma, Y. (2006). Robust statistical estimation and segmentation of multiple subspaces. In CVPR workshop on 25 years of RANSAC .

Cited By

Agudo ALepetit VMoreno-Noguer F(2022)Simultaneous completion and spatiotemporal grouping of corrupted motion tracksThe Visual Computer: International Journal of Computer Graphics10.1007/s00371-021-02238-838:11(3937-3952)Online publication date: 1-Nov-2022
https://dl.acm.org/doi/10.1007/s00371-021-02238-8
Abhadiomhen SWang ZShen XFan J(2021)Multiview Common Subspace Clustering via Coupled Low Rank RepresentationACM Transactions on Intelligent Systems and Technology10.1145/346505612:4(1-25)Online publication date: 1-Aug-2021
https://dl.acm.org/doi/10.1145/3465056
He WChen JZhang W(2017)Low-rank representation with graph regularization for subspace clusteringSoft Computing - A Fusion of Foundations, Methodologies and Applications10.1007/s00500-015-1869-021:6(1569-1581)Online publication date: 1-Mar-2017
https://dl.acm.org/doi/10.1007/s00500-015-1869-0
Show More Cited By

Robust Algebraic Segmentation of Mixed Rigid-Body and Planar Motions from Two Views
1. Computing methodologies
  1. Artificial intelligence
    1. Computer vision
      1. Computer vision problems
      2. Computer vision tasks
  2. Computer graphics
    1. Animation
2. Networks
  1. Network protocols

Recommendations

Robust uncalibrated stereo rectification with constrained geometric distortions (USR-CGD)

A novel algorithm for uncalibrated stereo image-pair rectification under the constraint of geometric distortion, called USR-CGD, is presented in this work. Although it is straightforward to define a rectifying transformation (or homography) given the ...
An Efficient Projective Rectification for Trinocular Stereo Vision
PDCAT '13: Proceedings of the 2013 International Conference on Parallel and Distributed Computing, Applications and Technologies

In this paper, we describe an efficient projective rectification method that can be applied to trinocular rectification to improve the visual appearance of stereo vision. This method expends the matrix-based correcting homographic approach which has ...
Camera Calibration With Three Noncollinear Points Under Special Motions

Plane-based (2D) camera calibration is becoming a hot research topic in recent years because of its flexibility. However, at least four image points are needed in every view to denote the coplanar feature in the 2D camera calibration. Can we do the ...

Comments

Information & Contributors

Information

Published In

cover image International Journal of Computer Vision

International Journal of Computer Vision Volume 88, Issue 3

July 2010

163 pages

ISSN:0920-5691

Issue’s Table of Contents

Copyright © Copyright © 2010 Springer Science+Business Media, LLC.

Publisher

Kluwer Academic Publishers

United States

Publication History

Published: 01 July 2010

Author Tags

Qualifiers

Article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

9
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 25 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Agudo ALepetit VMoreno-Noguer F(2022)Simultaneous completion and spatiotemporal grouping of corrupted motion tracksThe Visual Computer: International Journal of Computer Graphics10.1007/s00371-021-02238-838:11(3937-3952)Online publication date: 1-Nov-2022
https://dl.acm.org/doi/10.1007/s00371-021-02238-8
Abhadiomhen SWang ZShen XFan J(2021)Multiview Common Subspace Clustering via Coupled Low Rank RepresentationACM Transactions on Intelligent Systems and Technology10.1145/346505612:4(1-25)Online publication date: 1-Aug-2021
https://dl.acm.org/doi/10.1145/3465056
He WChen JZhang W(2017)Low-rank representation with graph regularization for subspace clusteringSoft Computing - A Fusion of Foundations, Methodologies and Applications10.1007/s00500-015-1869-021:6(1569-1581)Online publication date: 1-Mar-2017
https://dl.acm.org/doi/10.1007/s00500-015-1869-0
Liu MYan YChen RWang HWang HDavis LZhu WKopf SQu YYu JSang JMei T(2014)The State-of-the-Art Research Progress on Motion SegmentationProceedings of International Conference on Internet Multimedia Computing and Service10.1145/2632856.2632939(345-349)Online publication date: 10-Jul-2014
https://dl.acm.org/doi/10.1145/2632856.2632939
Poling BLerman G(2014)A New Approach to Two-View Motion Segmentation Using Global Dimension MinimizationInternational Journal of Computer Vision10.1007/s11263-013-0694-0108:3(165-185)Online publication date: 1-Jul-2014
https://dl.acm.org/doi/10.1007/s11263-013-0694-0
Li RLi BZhang KJin CXue XLangford JPineau J(2012)Groupwise constrained reconstruction for subspace clusteringProceedings of the 29th International Coference on International Conference on Machine Learning10.5555/3042573.3042597(155-162)Online publication date: 26-Jun-2012
https://dl.acm.org/doi/10.5555/3042573.3042597
Liu GLin ZYu Y(2010)Robust subspace segmentation by low-rank representationProceedings of the 27th International Conference on International Conference on Machine Learning10.5555/3104322.3104407(663-670)Online publication date: 21-Jun-2010
https://dl.acm.org/doi/10.5555/3104322.3104407
Kundu AKrishna KJawahar CChellappa RAnandan PRajagopalan ANarayanan PTorr P(2010)Realtime motion segmentation based multibody visual SLAMProceedings of the Seventh Indian Conference on Computer Vision, Graphics and Image Processing10.1145/1924559.1924593(251-258)Online publication date: 12-Dec-2010
https://dl.acm.org/doi/10.1145/1924559.1924593
Li JMo Shan Menglu Lan Yingcai Bi Hailong Qin Lin FChen B(undefined)Semi-dense motion segmentation for moving cameras by discrete energy minimizationIECON 2016 - 42nd Annual Conference of the IEEE Industrial Electronics Society10.1109/IECON.2016.7793862(6116-6121)
https://dl.acm.org/doi/10.1109/IECON.2016.7793862

View Options

View options

Figures

Tables

Media

View Issue’s Table of Contents