research-article

Acquiring 3D indoor environments with variability and repetition

Authors:

Niloy J. Mitra,

Leonidas GuibasAuthors Info & Claims

ACM Transactions on Graphics (TOG), Volume 31, Issue 6

Article No.: 138, Pages 1 - 11

https://doi.org/10.1145/2366145.2366157

Published: 01 November 2012 Publication History

Abstract

Large-scale acquisition of exterior urban environments is by now a well-established technology, supporting many applications in search, navigation, and commerce. The same is, however, not the case for indoor environments, where access is often restricted and the spaces are cluttered. Further, such environments typically contain a high density of repeated objects (e.g., tables, chairs, monitors, etc.) in regular or non-regular arrangements with significant pose variations and articulations. In this paper, we exploit the special structure of indoor environments to accelerate their 3D acquisition and recognition with a low-end handheld scanner. Our approach runs in two phases: (i) a learning phase wherein we acquire 3D models of frequently occurring objects and capture their variability modes from only a few scans, and (ii) a recognition phase wherein from a single scan of a new area, we identify previously seen objects but in different poses and locations at an average recognition time of 200ms/model. We evaluate the robustness and limits of the proposed recognition system using a range of synthetic and real world scans under challenging settings.

References

[1]

Besl, P. J., and McKay, N. D. 1992. A method for registration of 3-D shapes. IEEE PAMI 14, 2, 239--256.

Digital Library

[2]

Chang, W., and Zwicker, M. 2011. Global registration of dynamic range scans for articulated model reconstruction. ACM TOG 30, 3, 26:1--26:15.

Digital Library

[3]

Dey, T. K. 2007. Curve and Surface Reconstruction: Algorithms with Mathematical Analysis. Cambridge University Press.

Digital Library

[4]

Engelhard, N., Endres, F., Hess, J., Sturm, J., and Burgard, W. 2011. Real-time 3D visual SLAM with a hand-held RGB-D camera. In Proc. of the RGB-D Workshop on 3D Perception in Robotics at the European Robotics Forum.

[5]

Fisher, M., Savva, M., and Hanrahan, P. 2011. Characterizing structural relationships in scenes using graph kernels. ACM TOG 30, 4, 34:1--34:11.

Digital Library

[6]

Gupta, A., Efros, A. A., and Hebert, M. 2010. Blocks world revisited: Image understanding using qualitative geometry and mechanics. In ECCV, 482--496.

Digital Library

[7]

Henry, P., Krainin, M., Herbst, E., Ren, X., and Fox, D. 2010. RGB-D mapping: Using depth cameras for dense 3D modeling of indoor environments. In International Symposium on Experimental Robotics.

[8]

Huang, Q., Koltun, V., and Guibas, L. 2011. Joint-shape segmentation with linear programming. ACM TOG (SIGGRAPH Asia) 30, 6, 125:1--125:11.

Digital Library

[9]

Izadi, S., Kim, D., Hilliges, O., Molyneaux, D., Newcombe, R., Kohli, P., Shotton, J., Hodges, S., Freeman, D., Davison, A., and Fitzgibbon, A. 2011. Kinect-fusion: real-time 3D reconstruction and interaction using a moving depth camera. In Proc. UIST, 559--568.

Digital Library

[10]

Koppula, H., Anand, A., Joachims, T., and Saxena, A. 2011. Semantic labeling of 3D point clouds for indoor scenes. In NIPS, 244--252.

[11]

Lee, D. C., Gupta, A., Hebert, M., and Kanade, T. 2010. Estimating spatial layout of rooms using volumetric reasoning about objects and surfaces. In NIPS, 1288--1296.

[12]

Leordeanu, M., and Hebert, M. 2005. A spectral technique for correspondence problems using pairwise constraints. In ICCV, vol. 2, 1482--1489.

Digital Library

[13]

Li, H., Adams, B., Guibas, L. J., and Pauly, M. 2009. Robust single-view geometry and motion reconstruction. ACM TOG (SIGGRAPH) 28, 5, 175:1--175:10.

Digital Library

[14]

Mehra, R., Zhou, Q., Long, J., Sheffer, A., Gooch, A., and Mitra, N. J. 2009. Abstraction of man-made shapes. ACM TOG (SIGGRAPH Asia) 28, 5, #137, 1--10.

Digital Library

[15]

Mitra, N. J., Flory, S., Ovsjanikov, M., Gelfand, N., Guibas, L., and Pottmann, H. 2007. Dynamic geometry registration. In Symp. on Geometry Proc., 173--182.

Digital Library

[16]

Mitra, N., Yang, Y.-L., Yan, D.-M., Li, W., and Agrawala, M. 2010. Illustrating how mechanical assemblies work. ACM TOG (SIGGRAPH) 29, 4, 58:1--58:12.

Digital Library

[17]

Mitra, N. J., Pauly, M., Wand, M., and Ceylan, D. 2012. Symmetry in 3d geometry: Extraction and applications. In EUROGRAPHICS State-of-the-art Report.

[18]

Nan, L., Xie, K., and Sharf, A. 2012. A search-classify approach for cluttered indoor scene understanding. ACM TOG (SIGGRAPH Asia) 31, 6.

Digital Library

[19]

Ovsjanikov, M., Li, W., Guibas, L., and Mitra, N. J. 2011. Exploration of continuous variability in collections of 3D shapes. ACM TOG (SIGGRAPH) 30, 4, 33:1--33:10.

Digital Library

[20]

Pauly, M., Mitra, N. J., Giesen, J., Gross, M., and Guibas, L. J. 2005. Example-based 3D scan completion. In Symp. on Geometry Proc., 23--32.

Digital Library

[21]

Rusinkiewicz, S., Hall-Holt, O., and Levoy, M. 2002. Real-time 3D model acquisition. ACM TOG (SIGGRAPH) 21, 3, 438--446.

Digital Library

[22]

Schnabel, R., Wessel, R., Wahl, R., and Klein, R. 2008. Shape recognition in 3D point-clouds. In Proc. WSCG, 65--72.

[23]

Shao, T., Xu, W., Zhou, K., Wang, J., Li, D., and Guo, B. 2012. An interactive approach to semantic modeling of indoor scenes with an RGBD camera. ACM TOG (SIGGRAPH Asia) 31, 6.

Digital Library

[24]

Sidi, O., van Kaick, O., Kleiman, Y., Zhang, H., and Cohen-Or, D. 2011. Unsupervised co-segmentation of a set of shapes via descriptor-space spectral clustering. ACM TOG (SIGGRAPH Asia) 30, 6, 126:1--126:10.

Digital Library

[25]

Triebel, R., Shin, J., and Siegwart, R. 2010. Segmentation and unsupervised part-based discovery of repetitive objects. In Proceedings of Robotics: Science and Systems.

[26]

Xiang, Y., and Savarese, S. 2012. Estimating the aspect layout of object categories. In CVPR, 3410--3417.

Digital Library

[27]

Xu, K., Li, H., Zhang, H., Cohen-Or, D., Xiong, Y., and Cheng, Z. 2010. Style-content separation by anisotropic part scales. ACM TOG (SIGGRAPH Asia) 29, 5, 184:1--184:10.

Digital Library

[28]

Xu, K., Zheng, H., Zhang, H., Cohen-Or, D., Liu, L., and Xiong, Y. 2011. Photo-inspired model-driven 3D object modeling. ACM TOG (SIGGRAPH) 30, 4, 80:1--80:10.

Digital Library

[29]

Zheng, Y., Chen, X., Cheng, M.-M., Zhou, K., Hu, S.-M., and Mitra, N. J. 2012. Interactive images: Cuboid proxies for smart image manipulation. ACM TOG (SIGGRAPH) 31, 4, 99:1--99:11.

Digital Library

Cited By

Zhang WRahmani HYang XLiu JCai JKankanhalli MPrabhakaran BBoll SSubramanian RZheng LSingh VCesar PXie LXu D(2024)Reverse2Complete: Unpaired Multimodal Point Cloud Completion via Guided DiffusionProceedings of the 32nd ACM International Conference on Multimedia10.1145/3664647.3680590(5892-5901)Online publication date: 28-Oct-2024
https://dl.acm.org/doi/10.1145/3664647.3680590
Huang ZWen YWang ZRen JJia K(2024)Surface Reconstruction From Point Clouds: A Survey and a BenchmarkIEEE Transactions on Pattern Analysis and Machine Intelligence10.1109/TPAMI.2024.342920946:12(9727-9748)Online publication date: 1-Dec-2024
https://dl.acm.org/doi/10.1109/TPAMI.2024.3429209
Rozenberszki DLitany ODai A(2024)UnScene3D: Unsupervised 3D Instance Segmentation for Indoor Scenes2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)10.1109/CVPR52733.2024.01886(19957-19967)Online publication date: 16-Jun-2024
https://doi.org/10.1109/CVPR52733.2024.01886
Show More Cited By

Index Terms

Acquiring 3D indoor environments with variability and repetition
1. Computing methodologies
  1. Artificial intelligence
    1. Computer vision
      1. Image and video acquisition
        3D imaging
  2. Computer graphics
    1. Animation
    2. Shape modeling
2. Theory of computation
  1. Randomness, geometry and discrete structures
    1. Computational geometry

Recommendations

3D Face Recognition under Expressions, Occlusions, and Pose Variations

We propose a novel geometric framework for analyzing 3D faces, with the specific goals of comparing, matching, and averaging their shapes. Here we represent facial surfaces by radial curves emanating from the nose tips and use elastic shape analysis of ...
Shape analysis of local facial patches for 3D facial expression recognition

In this paper we address the problem of 3D facial expression recognition. We propose a local geometric shape analysis of facial surfaces coupled with machine learning techniques for expression classification. A computation of the length of the geodesic ...
Elastic radial curves to model 3D facial deformations
3DOR '10: Proceedings of the ACM workshop on 3D object retrieval

In this paper we study the challenging problem of 3D face recognition in presence of facial expressions. Therefore, the facial surfaces are represented by an indexed collections of radial curves on them, emanating from the nose tips. Facial shapes ...

Comments

Information & Contributors

Information

Published In

cover image ACM Transactions on Graphics

ACM Transactions on Graphics Volume 31, Issue 6

November 2012

794 pages

ISSN:0730-0301

EISSN:1557-7368

DOI:10.1145/2366145

Issue’s Table of Contents

Copyright © 2012 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 01 November 2012

Published in TOG Volume 31, Issue 6

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Funding Sources

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

137
Total Citations
View Citations
1,441
Total Downloads

Downloads (Last 12 months)49
Downloads (Last 6 weeks)5

Reflects downloads up to 28 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Zhang WRahmani HYang XLiu JCai JKankanhalli MPrabhakaran BBoll SSubramanian RZheng LSingh VCesar PXie LXu D(2024)Reverse2Complete: Unpaired Multimodal Point Cloud Completion via Guided DiffusionProceedings of the 32nd ACM International Conference on Multimedia10.1145/3664647.3680590(5892-5901)Online publication date: 28-Oct-2024
https://dl.acm.org/doi/10.1145/3664647.3680590
Huang ZWen YWang ZRen JJia K(2024)Surface Reconstruction From Point Clouds: A Survey and a BenchmarkIEEE Transactions on Pattern Analysis and Machine Intelligence10.1109/TPAMI.2024.342920946:12(9727-9748)Online publication date: 1-Dec-2024
https://dl.acm.org/doi/10.1109/TPAMI.2024.3429209
Rozenberszki DLitany ODai A(2024)UnScene3D: Unsupervised 3D Instance Segmentation for Indoor Scenes2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)10.1109/CVPR52733.2024.01886(19957-19967)Online publication date: 16-Jun-2024
https://doi.org/10.1109/CVPR52733.2024.01886
Akase ROkada Y(2024)3D Room Layout System Using IEC (Interactive Evaluational Computation)Encyclopedia of Computer Graphics and Games10.1007/978-3-031-23161-2_34(37-51)Online publication date: 5-Jan-2024
https://doi.org/10.1007/978-3-031-23161-2_34
Kim JMo KSung MWoo W(2023)Seg&Struct: The Interplay Between Part Segmentation and Structure Inference for 3D Shape Parsing2023 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV)10.1109/WACV56688.2023.00128(1226-1235)Online publication date: Jan-2023
https://doi.org/10.1109/WACV56688.2023.00128
Zhang STam HLi YMu TZhang S(2023)SceneViewer: Automating Residential Photography in Virtual EnvironmentsIEEE Transactions on Visualization and Computer Graphics10.1109/TVCG.2022.321483629:12(5523-5537)Online publication date: 1-Dec-2023
https://dl.acm.org/doi/10.1109/TVCG.2022.3214836
Zhang WZhou HDong ZLiu JYan QXiao C(2023)Point Cloud Completion Via Skeleton-Detail TransformerIEEE Transactions on Visualization and Computer Graphics10.1109/TVCG.2022.318524729:10(4229-4242)Online publication date: 1-Oct-2023
https://doi.org/10.1109/TVCG.2022.3185247
Xiangli YXu LPan XZhao NDai BLin D(2023)AssetField: Assets Mining and Reconfiguration in Ground Feature Plane Representation2023 IEEE/CVF International Conference on Computer Vision (ICCV)10.1109/ICCV51070.2023.00301(3228-3238)Online publication date: 1-Oct-2023
https://doi.org/10.1109/ICCV51070.2023.00301
Wu CJohnson JMalik JFeichtenhofer CGkioxari G(2023)Multiview Compressive Coding for 3D Reconstruction2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR)10.1109/CVPR52729.2023.00875(9065-9075)Online publication date: Jul-2023
https://doi.org/10.1109/CVPR52729.2023.00875
Wei XGu XSun JKoyejo SMohamed SAgarwal ABelgrave DCho KOh A(2022)Learning generalizable part-based feature representation for 3D point cloudsProceedings of the 36th International Conference on Neural Information Processing Systems10.5555/3600270.3602395(29305-29318)Online publication date: 28-Nov-2022
https://dl.acm.org/doi/10.5555/3600270.3602395
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Article

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Issue’s Table of Contents