Robust wide baseline scene alignment based on 3D viewpoint normalization
Pages 654 - 665
Abstract
This paper presents a novel scheme for automatically aligning two widely separated 3D scenes via the use of viewpoint invariant features. The key idea of the proposed method is following. First, a number of dominant planes are extracted in the SfM 3D point cloud using a novel method integrating RANSAC and MDL to describe the underlying 3D geometry in urban settings. With respect to the extracted 3D planes, the original camera viewing directions are rectified to form the front-parallel views of the scene. Viewpoint invariant features are extracted on the canonical views to provide a basis for further matching. Compared to the conventional 2D feature detectors (e.g. SIFT, MSER), the resulting features have following advantages: (1) they are very discriminative and robust to perspective distortions and viewpoint changes due to exploiting scene structure; (2) the features contain useful local patch information which allow for efficient feature matching. Using the novel viewpoint invariant features, wide-baseline 3D scenes are automatically aligned in terms of robust image matching. The performance of the proposed method is comprehensively evaluated in our experiments. It's demonstrated that 2D image feature matching can be significantly improved by considering 3D scene structure.
References
[1]
Hartley, R., Zisserman, A.: Multiple View Geometry in Computer Vision. Cambridge University Press, Cambridge (2003)
[2]
Snavely, N., Seitz, S.M., Szeliski, R.: Modeling the world from Internet photo collections. IJCV 80(2), 189-210 (2008)
[3]
Pollefeys, M., Van Gool, L., Vergauwen, M., Verbiest, F., Cornelis, K., Tops, J., Koch, R.: Visual modeling with a hand-held camera. IJCV 59(3), 207-232 (2004)
[4]
Bay, H., Ess, A., Tuytelaars, T., Van Gool, L.: Speeded-up robust features (SURF). Comput. Vis. Image Underst. 110(3), 346-359 (2008)
[5]
Tuytelaars, T., Van Gool, L.: Matching widely separated views based on affine invariant regions. IJCV 59(1), 61-85 (2004)
[6]
Matas, J., Chum, O., Urban, M., Pajdla, T.: Robust wide baseline stereo from maximally stable extremal regions. In: BMVC (2002)
[7]
Lowe, D.G.: Distinctive image features from scale-invariant keypoints. IJCV 60(2), 91-110 (2004)
[8]
Donoser, M., Bischof, H.: Efficient maximally stable extremal region (MSER) tracking. In: CVPR, pp. 553-560 (2006)
[9]
Mikolajczyk, K., Schmid, C.: Scale and affine invariant interest point detectors. IJCV 60(1), 63-86 (2004)
[10]
Mikolajczyk, K., Tuytelaars, T., Schmid, C., Zisserman, A., Matas, J., Schaffalitzky, F., Kadir, T., Van Gool, L.: A comparison of affine region detectors. IJCV 65(1-2), 43-72 (2005)
[11]
Sinha, S., Steedly, D., Szeliski, R.: Piecewise planar stereo for image-based rendering. In: ICCV, pp. 1881-1888 (2009)
[12]
Furukawa, Y., Curless, B., Seitz, S., Szeliski, R.: Manhattan-world stereo. In: CVPR, pp. 1422-1429 (2009)
[13]
Wu, C., Clipp, B., Li, X., Frahm, J., Pollefeys, M.: 3d model matching with viewpoint-invariant patches (VIP). In: CVPR, pp. 1-8 (2008)
[14]
Koeser, K., Koch, R.: Perspectively invariant normal features. In: ICCV, pp. 14-21 (2007)
[15]
Besl, P., McKay, N.: A method for registration of 3-d shapes. PAMI 14(2), 239-256 (1992)
[16]
Zhao, W., Nister, D., Hsu, S.: Alignment of continuous video onto 3d point clouds. PAMI 27(8), 1305-1318 (2005)
[17]
Pottmann, H., Huang, Q., Yang, Y., Hu, S.: Geometry and convergence analysis of algorithms for registration of 3d shapes. IJCV 67(3), 277-296 (2006)
[18]
Seo, J., Sharp, G., Lee, S.: Range data registration using photometric features. In: CVPR II, pp. 1140-1145 (2005)
[19]
Liu, L., Stamos, I., Yu, G., Zokai, S.: Multiview geometry for texture mapping 2d images onto 3d range data. In: CVPR II, pp. 2293-2300 (2006)
[20]
Ikeuchi, K., Oishi, T., Takamatsu, J., Sagawa, R., Nakazawa, A., Kurazume, R., Nishino, K., Kamakura, M., Okamoto, Y.: The great buddha project: Digitally archiving, restoring, and analyzing cultural heritage objects. IJCV 75(1), 189-208 (2007)
[21]
Gonzalez Aguilera, D., Rodriguez Gonzalvez, P., Gomez Lahoz, J.: An automatic procedure for co-registration of terrestrial laser scanners and digital cameras. ISPRS Journal of Photogrammetry and Remote Sensing 64(3), 308-316 (2009)
[22]
Mikolajczyk, K., Schmid, C.: A performance evaluation of local descriptors. PAMI 27(10), 1615-1630 (2005)
[23]
Fischler, M., Bolles, R.: Random sample consensus: A paradigm for model fitting with applications to image analysis and automated cartography. Comm. of the ACM 24(6), 381-395 (1981)
[24]
Pan, H.: Two-level global optimization for image segmentation. ISPRS Journal of Photogrammetry and Remote Sensing 49, 21-32 (1994)
[25]
Rissanen, J.: Modelling by shortest data description. Automatica 14, 465-471 (1978)
[26]
Zhang, W., Košecká, J.: Hierarchical building recognition. Image Vision Comput. 25(5), 704-716 (2007)
[27]
Läbe, T., Förstner, W.: Automatic relative orientation of images. In: Proceedings of the 5th Turkish-German Joint Geodetic Days (2006)
[28]
Furukawa, Y., Ponce, J.: Accurate, dense, and robust multi-view stereopsis. PAMI (2009)
Recommendations
Robust Wide Baseline Scene Alignment Based on 3D Viewpoint Normalization
Advances in Visual ComputingAbstractThis paper presents a novel scheme for automatically aligning two widely separated 3D scenes via the use of viewpoint invariant features. The key idea of the proposed method is following. First, a number of dominant planes are extracted in the SfM ...
Wide-baseline stereo vision for terrain mapping
Integrated Imaging and Vision Techniques for Industrial InspectionTerrain mapping is important for mobile robots to perform localization and navigation. Stereo vision has been used extensively for this purpose in outdoor mapping tasks. However, conventional stereo does not scale well to distant terrain. This paper ...
ROVO: Robust Omnidirectional Visual Odometry for Wide-baseline Wide-FOV Camera Systems
2019 International Conference on Robotics and Automation (ICRA)In this paper we propose a robust visual odometry system for a wide-baseline camera rig with wide field-of-view (FOV) fisheye lenses, which provides full omnidirectional stereo observations of the environment. For more robust and accurate ego-motion ...
Comments
Information & Contributors
Information
Published In
November 2010
759 pages
ISBN:3642172881
Sponsors
- NASA: National Aeronatics and Space Administration
- DigitalPersona
- Intel: Intel
- Equinox Corporation
- HP invent
Publisher
Springer-Verlag
Berlin, Heidelberg
Publication History
Published: 29 November 2010
Qualifiers
- Article
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 0Total Downloads
- Downloads (Last 12 months)0
- Downloads (Last 6 weeks)0
Reflects downloads up to 25 Feb 2025