Article

Exploring high-level plane primitives for indoor 3d reconstruction with a hand-held RGB-D camera

Authors:

Jan-Michael Frahm,

Henry FuchsAuthors Info & Claims

ACCV'12: Proceedings of the 11th international conference on Computer Vision - Volume 2

Pages 94 - 108

https://doi.org/10.1007/978-3-642-37484-5_9

Published: 05 November 2012 Publication History

Abstract

Given a hand-held RGB-D camera (e.g. Kinect), methods such as Structure from Motion (SfM) and Iterative Closest Point (ICP), perform poorly when reconstructing indoor scenes with few image features or little geometric structure information. In this paper, we propose to extract high level primitives---planes---from an RGB-D camera, in addition to low level image features (e.g. SIFT), to better constrain the problem and help improve indoor 3D reconstruction. Our work has two major contributions: first, for frame to frame matching, we propose a new scheme which takes into account both low-level appearance feature correspondences in RGB image and high-level plane correspondences in depth image. Second, in the global bundle adjustment step, we formulate a novel error measurement that not only takes into account the traditional 3D point re-projection errors, but also the planar surface alignment errors. We demonstrate with real datasets that our method with plane constraints achieves more accurate and more appealing results comparing with other state-of-the-art scene reconstruction algorithms in aforementioned challenging indoor scenarios.

References

[1]

Henry, P., Krainin, M., Ren, X., Herbrt, E., Fox, D.: Rgb-d mapping: Using depth cameras for dense 3d modeling of indoor environments. ISER (2010)

[2]

Newcombe, R. A., Izadi, S., Hilliges, O., Molyneaux, D., Kim, D., Davison, A. J., Kohli, P., Shotton, J., Hodges, S., Fitzgibbon, A.: Kinectfusion: Real-time dense surface mapping and tracking. In: IEEE ISMAR (2011)

Digital Library

[3]

Neumann, D., Lugauer, F., Bauer, S., Wasza, J., Hornegger, J.: Real-time rgb-d mapping on the gpu using the random ball cover data structure. In: IEEE ICCV/CDC4CV (2011)

[4]

Lieberknecht, S., Huber, A., Ilic, S., Benhimane, S.: Rgb-d camera-based parallel tracking and meshing. In: ISMAR (2011)

Digital Library

[5]

Snavely, N., Seitz, S. M., Szeliski, R.: Modeling the world from internet photo collections. International Journal of Computer Vision (2007)

Digital Library

[6]

Snavely, N., Seitz, S. M., Szeliski, R.: Photo tourism: Exploring image collections in 3d. ACM Transactions on Graphics (2006)

Digital Library

[7]

Crandall, D., Owens, A., Snavely, N., Huttenlocher, D. P.: Discrete-continuous optimization for large-scale structure from motion. In: CVPR (2011)

Digital Library

[8]

Sinha, S. N., Steedly, D., Szeliski, R.: Piecewise planar stereo for image-based rendering. In: ICCV (2009)

[9]

Furukawa, Y., Curless, B., Seitz, S. M., Szeliski, R.: Manhattan-world stereo. In: CVPR (2009)

[10]

Gallup, D., Frahm, J.-M.: Piecewise planar and non-planar stereo for urban scene reconstruction. In: CVPR (2010)

[11]

Lee, G. H., Fraundorfer, F., Pollefeys, M.: Mav visual slam with plane constraint. In: IEEE Int. Conf. on Robotics and Automation (2011)

[12]

Pathak, K., Birk, A., Vaskevicius, N., Poppinga, J.: Fast registration based on noisy planes with unknown correspondesces for 3-d mapping. IEEE Transactions on Robotics (2010)

Digital Library

[13]

Pathak, K., Vaskevicius, N., Poppinga, J., Pfingsthorn, M., Schwertfeger, S., Birk, A.: Fast 3d mapping by matching planes extracted from range sensor point-clouds. IROS (2009)

Digital Library

[14]

Pollefeys, M., Van Gool, L., Vergauwen, M., Verbiest, F., Cornelis, K., Tops, J., Koch, R.: Visual modeling with a hand-held camera. IJCV (2004)

Digital Library

[15]

Steffen, R., Frahm, J.-M., Förstner, W.: Relative Bundle Adjustment Based on Trifocal Constraints. In: Kutulakos, K. N. (ed.) ECCV 2010 Workshops, Part II. LNCS, vol. 6554, pp. 282-295. Springer, Heidelberg (2012)

Digital Library

[16]

Poppinga, J., Vaskevicius, N., Birk, A., Pathak, K.: Fast plane detection and polygonalization in noisy 3d range images. IROS (2008)

[17]

Borrmann, D., Elseberg, J., Lingemann, K., Nuhter, A.: The 3d hough transform for plane detection in point clouds: A review and a new accumulator design. 3D Research 02, 1330- 1334 (2011)

Digital Library

[18]

Raguram, R., Frahm, J.-M., Pollefeys, M.: A Comparative Analysis of RANSAC Techniques Leading to Adaptive Real-Time Random Sample Consensus. In: Forsyth, D., Torr, P., Zisserman, A. (eds.) ECCV 2008, Part II. LNCS, vol. 5303, pp. 500-513. Springer, Heidelberg (2008)

Digital Library

[19]

Fischler, M. A., Bolles, R.C.: Random sample consensus: A paradigm for model fitting with applications to image analysis and automated cartography. Comm. of the ACM 24 (1981)

Digital Library

[20]

Lourakis, M. I. A.: Sparse Non-linear Least Squares Optimization for Geometric Vision. In: Daniilidis, K., Maragos, P., Paragios, N. (eds.) ECCV 2010, Part II. LNCS, vol. 6312, pp. 43-56. Springer, Heidelberg (2010)

Digital Library

[21]

Grisetti, G., Grzonka, S., Stachniss, C., Pfaff, P., Burgard, W.: Efficient estimation of accurate maximum likelihood maps in 3d. IROS (2007)

Cited By

Wang RZha WMeng XMeng FWu YGe JGu D(2020)Semantic Ground Plane Constraint in Visual SLAM for Indoor ScenesPattern Recognition and Computer Vision10.1007/978-3-030-60633-6_22(268-279)Online publication date: 16-Oct-2020
https://dl.acm.org/doi/10.1007/978-3-030-60633-6_22
Shi YLong PXu KHuang HXiong Y(2018)Data-driven contextual modeling for 3D scene understandingComputers and Graphics10.1016/j.cag.2015.11.00355:C(55-67)Online publication date: 23-Dec-2018
https://dl.acm.org/doi/10.1016/j.cag.2015.11.003
Shi YXu KNießner MRusinkiewicz SFunkhouser T(2018)PlaneMatch: Patch Coplanarity Prediction for Robust RGB-D ReconstructionComputer Vision – ECCV 201810.1007/978-3-030-01237-3_46(767-784)Online publication date: 8-Sep-2018
https://dl.acm.org/doi/10.1007/978-3-030-01237-3_46
Show More Cited By

Exploring high-level plane primitives for indoor 3d reconstruction with a hand-held RGB-D camera
1. Computing methodologies
  1. Artificial intelligence
    1. Computer vision
2. Networks
  1. Network protocols

Recommendations

Realistic surface geometry reconstruction using a hand-held RGB-D camera

In this paper, we have proposed a novel approach for the reconstruction of real object/scene with realistic surface geometry using a hand-held, low-cost, RGB-D camera. To achieve accurate reconstruction, the most important issues to consider are the ...
Dense 3-D Reconstruction of an Outdoor Scene by Hundreds-Baseline Stereo Using a Hand-Held Video Camera

Three-dimensional (3-D) models of outdoor scenes are widely used for object recognition, navigation, mixed reality, and so on. Because such models are often made manually with high costs, automatic 3-D reconstruction has been widely investigated. In ...
Dense 3D reconstruction combining depth and RGB information

Dense 3D reconstruction has important applications in many fields. The existing depth information based methods are typically constrained in their effective camera-object distance which should be from 0.4m to 4m. We present a novel method that can ...

Comments

Information & Contributors

Information

Published In

cover image Guide Proceedings

ACCV'12: Proceedings of the 11th international conference on Computer Vision - Volume 2

November 2012

605 pages

ISBN:9783642374838

Editors:
Jong-Il Park
Computer Science and Engineering, Hanyang University, 222 Wangshimni-ro, Seongdong-gu, Seoul, South Korea
,
Junmo Kim
Department of Electrical Engineering, KAIST, 291 Daehak-ro, Yuseong-gu, Daejeon, South Korea

Publisher

Springer-Verlag

Berlin, Heidelberg

Publication History

Published: 05 November 2012

Qualifiers

Article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

8
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 07 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Wang RZha WMeng XMeng FWu YGe JGu D(2020)Semantic Ground Plane Constraint in Visual SLAM for Indoor ScenesPattern Recognition and Computer Vision10.1007/978-3-030-60633-6_22(268-279)Online publication date: 16-Oct-2020
https://dl.acm.org/doi/10.1007/978-3-030-60633-6_22
Shi YLong PXu KHuang HXiong Y(2018)Data-driven contextual modeling for 3D scene understandingComputers and Graphics10.1016/j.cag.2015.11.00355:C(55-67)Online publication date: 23-Dec-2018
https://dl.acm.org/doi/10.1016/j.cag.2015.11.003
Shi YXu KNießner MRusinkiewicz SFunkhouser T(2018)PlaneMatch: Patch Coplanarity Prediction for Robust RGB-D ReconstructionComputer Vision – ECCV 201810.1007/978-3-030-01237-3_46(767-784)Online publication date: 8-Sep-2018
https://dl.acm.org/doi/10.1007/978-3-030-01237-3_46
Kaiser AYbanez Zepeda JBoubekeur T(2018)Proxy Clouds for Live RGB-D Stream Processing and ConsolidationComputer Vision – ECCV 201810.1007/978-3-030-01231-1_16(255-271)Online publication date: 8-Sep-2018
https://dl.acm.org/doi/10.1007/978-3-030-01231-1_16
Huang JDai AGuibas LNiessner M(2017)3DliteACM Transactions on Graphics10.1145/3130800.313082436:6(1-14)Online publication date: 20-Nov-2017
https://dl.acm.org/doi/10.1145/3130800.3130824
Bobenrieth CSeo HHabibi ACordier FMao XThalmann DGavrilova M(2017)Indoor scene reconstruction from a sparse set of 3D shotsProceedings of the Computer Graphics International Conference10.1145/3095140.3095167(1-5)Online publication date: 27-Jun-2017
https://dl.acm.org/doi/10.1145/3095140.3095167
Lai PLaganière R(2017)Creating Immersive Virtual Reality Scenes Using a Single RGB-D CameraImage Analysis and Recognition10.1007/978-3-319-59876-5_25(221-230)Online publication date: 5-Jul-2017
https://dl.acm.org/doi/10.1007/978-3-319-59876-5_25
Zhang YXu WTong YZhou K(2015)Online Structure Analysis for Real-Time Indoor Scene ReconstructionACM Transactions on Graphics10.1145/276882134:5(1-13)Online publication date: 3-Nov-2015
https://dl.acm.org/doi/10.1145/2768821

View Options

View options

Figures

Tables

Media

View Table of Conten