article

A Theory of Shape by Space Carving

Authors:

Kiriakos N. Kutulakos,

Steven M. SeitzAuthors Info & Claims

International Journal of Computer Vision, Volume 38, Issue 3

Pages 199 - 218

https://doi.org/10.1023/A:1008191222954

Published: 21 July 2000 Publication History

Abstract

In this paper we consider the problem of computing the 3D shape of an unknown, arbitrarily-shaped scene from multiple photographs taken at known but arbitrarily-distributed viewpoints. By studying the equivalence class of all 3D shapes that reproduce the input photographs, we prove the existence of a special member of this class, the photo hull, that (1) can be computed directly from photographs of the scene, and (2) subsumes all other members of this class. We then give a provably-correct algorithm, called Space Carving, for computing this shape and present experimental results on complex real-world scenes. The approach is designed to (1) capture photorealistic shapes that accurately model scene appearance from a wide range of viewpoints, and (2) account for the complex interactions between occlusion, parallax, shading, and their view-dependent effects on scene-appearance.

References

[1]

Alfvin, R.L. and Fairchild, M.D. 1997. Observer variability in metameric color matches using color reproduction media. Color Research & Application, 22(3):174-178.

[2]

Aloimonos, Y. 1988. Visual shape computation. Proc. IEEE, 76:899-916.

[3]

Armstrong, M.A. 1983. Basic Topology. Springer-Verlag.

[4]

Bascle, B. and Deriche, R. 1993. Stereo matching, reconstruction and refinement of 3D curves using deformable contours. In Proc. 4th Int. Conf. Computer Vision, pp. 421-430.

[5]

Beardsley, P., Torr, P., and Zisserman, A. 1996. 3D model acquisition from extended image sequences. In Proc. 4th European Conf. on Computer Vision, pp. 683-695.

Digital Library

[6]

Belhumeur, P.N. 1996. A bayesian approach to binocular stereopsis. Int. J. on Computer Vision, 19(3):237-260.

Digital Library

[7]

Belhumeur, P.N. and Kriegman, D.J. 1996. What is the set of images of an object under all possible lighting conditions? In Proc. Computer Vision and Pattern Recognition, pp. 270-277.

Digital Library

[8]

Bolles, R.C., Baker, H.H., and Marimont, D.H. 1987. Epipolar-plane image analysis: An approach to determining structure from motion. Int. J. Computer Vision, 1:7-55.

[9]

Bolles, R.C. and Cain, R.A. 1982. Recognizing and locating partially-visible objects: The local-feature-focus method. Int. J. Robotics Research, 1(3):57-82.

[10]

Cipolla, R. and Blake, A. 1992. Surface shape from the deformation of apparent contours. Int. J. Computer Vision, 9(2):83-112.

Digital Library

[11]

Collins, R.T. 1996. A space-sweep approach to true multiimage matching. In Proc. Computer Vision and Pattern Recognition Conf., pp. 358-363.

Digital Library

[12]

Cox, I., Hingorani, S., Rao, S., and Maggs, B. 1996. A maximum likelihood stereo algorithm. CVIU: Image Understanding, 63(3):542-567.

Digital Library

[13]

Culbertson, W.B., Malzbender, T., and Slabaugh, G. 1999. Generalized voxel coloring. In Workshop on Vision Algorithms: Theory and Practice, Corfu, Greece.

[14]

Curless, B. and Levoy, M. 1996. A volumetric method for building complex models from range images. In Proc. SIGGRAPH'96, pp. 303-312.

Digital Library

[15]

Debevec, P.E., Taylor, C.J., and Malik, J. 1996. Modeling and rendering architecture from photographs: A hybrid geometry- and image-based approach. In Proc. SIGGRAPH'96, pp. 11-20.

Digital Library

[16]

Epstein, R., Yuille, A.L., and Belhumeur, P.N. 1996. Learning object representations from lighting variations. In Object Representation in Computer Vision II, J. Ponce, A. Zisserman, and M. Hebert (Eds.). Springer-Verlag, pp. 179-199.

Digital Library

[17]

Faugeras, O. 1995. Stratification of three-dimensional vision: Projective, affine, and metric representations. J. Opt. Soc. Am. A, 12(3):465-484.

[18]

Faugeras, O.D. 1998. Personal communication.

[19]

Faugeras, O. and Keriven, R. 1998. Complete dense stereovision using level set methods. In Proc. 5th European Conf. on Computer Vision, pp. 379-393.

Digital Library

[20]

Faugeras, O.D. and Maybank, S. 1990. Motion from point matches: Multiplicity of solutions. Int. J. Computer Vision, 4:225-246.

Digital Library

[21]

Foley, J.D., van Dam, A., Feiner, S.K., and Hughes, J.F. 1990. Computer Graphics Principles and Practice. Addison-Wesley Publishing Co.

Digital Library

[22]

Forsyth, D. and Zisserman, A. 1991. Reflections on shading. IEEE Trans. Pattern Anal. Machine Intell., 13(7):671-679.

Digital Library

[23]

Freund, J.E. 1992. Mathematical Statistics. Prentice Hall: Englewood Cliffs, NJ.

Digital Library

[24]

Fua, P. and Leclerc, Y.G. 1995. Object-centered surface reconstruction: Combining multi-image stereo and shading. Int. J. Computer Vision, 16:35-56.

Digital Library

[25]

Fuchs, H., Kedem, Z., and Naylor, B.F. 1980. On visible surface generation by a priori tree structures. In Proc. SIGGRAPH '80, pp. 39-48.

Digital Library

[26]

Hoff, W. and Ahuja, N. 1989. Surfaces from stereo: Integrating feature matching, disparity estimation, and contour detection. IEEE Trans. Pattern Anal. Machine Intell., 11:121-136.

Digital Library

[27]

Kakadiaris, I.A. and Metaxas, D. 1995. 3D human body model acquisition from multiple views. In Proc. Int. Conf. on Computer Vision, pp. 618-623.

Digital Library

[28]

Kanade, T., Narayanan, P.J., and Rander, P.W. 1995. Virtualized reality: Concepts and early results. In Proc. Workshop on Representations of Visual Scenes, pp. 69-76.

Digital Library

[29]

Kanade, T., Yoshida, A., Oda, K., Kano, H., and Tanaka, M. 1996. A stereo machine for video-rate dense depth mapping and its new applications. In Proc. Computer Vision and Pattern Recognition Conf.

Digital Library

[30]

Kang, S.B. and Szeliski, R. 1996. 3-D scene data recovery using omnidirectional multibaseline stereo. In Proc. Computer Vision and Pattern Recognition Conf., pp. 364-370.

Digital Library

[31]

Katayama, A., Tanaka, K., Oshino, T., and Tamura, H. 1995. A viewpoint dependent stereoscopic display using interpolation of multi-viewpoint images. In Proc. SPIE, Vol. 2409A, pp. 21-30.

[32]

Koenderink, J.J. and van Doorn, A.J. 1991. Affine structure from motion. J. Opt. Soc. Am., A(2):377-385.

[33]

Kutulakos, K.N. 1997. Shape from the light field boundary. In Proc. Computer Vision and Pattern Recognition, pp. 53-59.

Digital Library

[34]

Kutulakos, K.N. 2000. Approximate N-view stereo. In Proc. European Conf. on Computer Vision.

[35]

Kutulakos, K.N. and Dyer, C.R. 1994. Recovering shape by purposive viewpoint adjustment. Int. J. Computer Vision, 12(2):113- 136.

Digital Library

[36]

Kutulakos, K.N. and Dyer, C.R. 1995. Global surface reconstruction by purposive control of observer motion. Artificial Intelligence Journal, 78(1-2):147-177.

Digital Library

[37]

Langer, M.S. and Zucker, S.W. 1994. Shape-from-shading on a cloudy day. J. Opt. Soc. Am. A, 11(2):467-478.

[38]

Laurentini, A. 1994. The visual hull concept for silhouette-based image understanding. IEEE Trans. Pattern Anal. Machine Intell., 16(2):150-162.

Digital Library

[39]

Marr, D. 1982. Vision. Freeman.

[40]

Martin, W.N. and Aggarwal, J.K. 1983. Volumetric descriptions of objects from multiple views. IEEE Proc. Pattern Anal. Machine Intell., 5(2):150-158.

Digital Library

[41]

Moezzi, S., Katkere, A., Kuramura, D.Y., and Jain, R. 1996. Reality modeling and visualization from multiple video sequences. IEEE Computer Graphics and Applications, 16(6):58-63.

Digital Library

[42]

Mundy, J.L. and Zisserman, A. (Eds.). 1992. Geometric Invariance in Computer Vision. MIT Press.

Digital Library

[43]

Narayanan, P.J., Rander, P.W., and Kanade, T. 1998. Constructing virtual worlds using dense stereo. In Proc. Int. Conf. on Computer Vision, pp. 3-10.

Digital Library

[44]

Newell, M.E., Newell, R.G., and Sancha, T.L. 1972. A solution to the hidden surface problem. In Proc. ACM National Conference, pp. 443-450.

Digital Library

[45]

Okutomi, M. and Kanade, T. 1993. A multiple-baseline stereo. IEEE Trans. Pattern Anal. Machine Intell., 15(4):353-363.

Digital Library

[46]

Poggio, T., Torre, V., and Koch, C. 1985. Computational vision and regularization theory. Nature, 317(26):314-319.

Digital Library

[47]

Pollefeys, M., Koch, R., and Gool, L.V. 1998. Self-calibration and metric reconstruction in spite of varying and unknown internal camera parameters. In Proc. 6th Int. Conf. on Computer Vision, pp. 90-95.

Digital Library

[48]

Pritchett, P. and Zisserman, A. 1998. Wide baseline stereo matching. In Proc. 6th Int. Conf. on Computer Vision, pp. 754-760.

Digital Library

[49]

Roy, S. and Cox, I.J. 1998. A maximum-flow formulation of the N-camera stereo correspondence problem. In Proc. 6th Int. Conf. on Computer Vision, pp. 492-499.

Digital Library

[50]

Sato, Y., Wheeler, M.D., and Ikeuchi, K. 1997. Object shape and reflectance modeling from observation. In Proc. SIGGRAPH'97, pp. 379-387.

Digital Library

[51]

Seales, W.B. and Faugeras, O. 1995. Building three-dimensional object models from image sequences. Computer Vision and Image Understanding, 61(3):308-324.

Digital Library

[52]

Seitz, S.M. and Dyer, C.R. 1995. Complete scene structure from four point correspondences. In Proc. 5th Int. Conf. on Computer Vision, pp. 330-337.

Digital Library

[53]

Seitz, S.M. and Dyer, C.R. 1999. Photorealistic scene reconstruction by voxel coloring. Int. J. Computer Vision, 35(2):151- 173.

Digital Library

[54]

Seitz, S.M. and Kutulakos, K.N. 1998. Plenoptic image editing. In Proc. 6th Int. Conf. Computer Vision, pp. 17-24.

Digital Library

[55]

Smith, A.R. and Blinn, J.F. 1996. Blue screen matting. In Proc. SIGGRAPH'96, pp. 259-268.

Digital Library

[56]

Stewart, C.V. 1995. MINPRAN: A new robust estimator for computer vision. IEEE Trans. Pattern Anal. Machine Intell., 17(10): 925-938.

Digital Library

[57]

Szeliski, R. 1993. Rapid octree construction from image sequences. CVGIP: Image Understanding, 58(1):23-32.

Digital Library

[58]

Szeliski, R. and Golland, P. 1998. Stereo matching with transparency and matting. In Proc. 6th Int. Conf. on Computer Vision, pp. 517- 524.

Digital Library

[59]

Szeliski, R. and Weiss, R. 1994. Robust shape recovery from occluding contours using a linear smoother. In Real-time Computer Vision, C.M. Brown and D. Terzopoulos (Eds.). Cambridge University Press, pp. 141-165.

Digital Library

[60]

Tomasi, C. and Kanade, T. 1992. Shape and motion from image streams under orthography: A factorization method. Int. J. Computer Vision, 9(2):137-154.

Digital Library

[61]

Torrance, K.E. and Sparrow, E.M. 1967. Theory of off-specular reflection from roughened surface. Journal of the Optical Society of America, 57:1105-1114.

[62]

Turk, G. and Levoy, M. 1994. Zippered polygon meshes from range images. In Proc. SIGGRAPH'94, pp. 311-318.

Digital Library

[63]

Vaillant, R. and Faugeras, O.D. 1992. Using extremal boundaries for 3-D object modeling. IEEE Trans. Pattern Anal. Machine Intell., 14(2):157-173.

Digital Library

[64]

van Veen, J.A.J.C. and Werkhoven, P. 1996. Metamerisms in structure-from-motion perception. Vision Research, 36(14):2197- 2210.

[65]

Woodham, R.J., Iwahori, Y., and Barman, R.A. 1991. Photometric stereo: Lambertian reflectance and light sources with unknown direction and strength. Technical Report 91-18, University of British Columbia, Laboratory for Computational Intelligence.

Digital Library

[66]

Zhang, Z. 1998. Image-based geometrically-correct photorealistic scene/object modeling (IBPhM): A review. In Proc. 3rd Asian Conf. on Computer Vision, pp. 340-349.

Digital Library

[67]

Zhao, C. and Mohr, R. 1996. Global three-dimensional surface reconstruction from occluding contours. Computer Vision and Image Understanding, 64(1):62-96.

Digital Library

[68]

Zitnick, C.L. and Webb, J.A. 1996. Multi-baseline stereo using surface extraction. Technical Report CMU-CS-96-196, Carnegie Mellon University, Pittsburgh, PA.

Cited By

Liu YWang LYang JChen WMeng XYang BGao L(2024)NeUDF: Learning Neural Unsigned Distance Fields With Volume RenderingIEEE Transactions on Pattern Analysis and Machine Intelligence10.1109/TPAMI.2023.333535346:4(2364-2377)Online publication date: 1-Apr-2024
https://dl.acm.org/doi/10.1109/TPAMI.2023.3335353
Shao MXia CDuan DWang X(2024)Polarimetric Inverse Rendering for Transparent Shapes ReconstructionIEEE Transactions on Multimedia10.1109/TMM.2024.337179226(7801-7811)Online publication date: 29-Feb-2024
https://dl.acm.org/doi/10.1109/TMM.2024.3371792
Chang JHe JZhang TYu JWu F(2024)EI-MVSNet: Epipolar-Guided Multi-View Stereo Network With Interval-Aware LabelIEEE Transactions on Image Processing10.1109/TIP.2023.334792933(753-766)Online publication date: 1-Jan-2024
https://dl.acm.org/doi/10.1109/TIP.2023.3347929
Show More Cited By

Index Terms

A Theory of Shape by Space Carving
1. Computing methodologies
  1. Artificial intelligence
    1. Computer vision
  2. Computer graphics
    1. Shape modeling

Recommendations

A Theory of Refractive and Specular 3D Shape by Light-Path Triangulation

We investigate the feasibility of reconstructing an arbitrarily-shaped specular scene (refractive or mirror-like) from one or more viewpoints. By reducing shape recovery to the problem of reconstructing individual 3D light paths that cross the image ...
Appearance-Cloning: Photo-Consistent Scene Recovery from Multi-View Images

This paper introduces the novel volumetric methodology "appearance-cloning" as a viable solution for achieving a more improved photo-consistent scene recovery, including a greatly enhanced geometric recovery performance, from a set of photographs taken ...
Methods for Volumetric Reconstruction of Visual Scenes

In this paper, we present methods for 3D volumetric reconstruction of visual scenes photographed by multiple calibrated cameras placed at arbitrary viewpoints. Our goal is to generate a 3D model that can be rendered to synthesize new photo-realistic ...

Comments

Information & Contributors

Information

Published In

cover image International Journal of Computer Vision

International Journal of Computer Vision Volume 38, Issue 3

Special issue on Genomic Signal Processing

July-August 2000

67 pages

ISSN:0920-5691

Issue’s Table of Contents

Copyright © Copyright © 2000 Kluwer Academic Publishers.

Publisher

Kluwer Academic Publishers

United States

Publication History

Published: 21 July 2000

Author Tags

Qualifiers

Article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

201
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 09 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Liu YWang LYang JChen WMeng XYang BGao L(2024)NeUDF: Learning Neural Unsigned Distance Fields With Volume RenderingIEEE Transactions on Pattern Analysis and Machine Intelligence10.1109/TPAMI.2023.333535346:4(2364-2377)Online publication date: 1-Apr-2024
https://dl.acm.org/doi/10.1109/TPAMI.2023.3335353
Shao MXia CDuan DWang X(2024)Polarimetric Inverse Rendering for Transparent Shapes ReconstructionIEEE Transactions on Multimedia10.1109/TMM.2024.337179226(7801-7811)Online publication date: 29-Feb-2024
https://dl.acm.org/doi/10.1109/TMM.2024.3371792
Chang JHe JZhang TYu JWu F(2024)EI-MVSNet: Epipolar-Guided Multi-View Stereo Network With Interval-Aware LabelIEEE Transactions on Image Processing10.1109/TIP.2023.334792933(753-766)Online publication date: 1-Jan-2024
https://dl.acm.org/doi/10.1109/TIP.2023.3347929
Yin HLuo STang JHua J(2024)RGB camera-based monocular stereo vision applied in plant phenotypeComputers and Electronics in Agriculture10.1016/j.compag.2024.109523227:P1Online publication date: 1-Dec-2024
https://dl.acm.org/doi/10.1016/j.compag.2024.109523
Sundt PTheoharis T(2024)Towards multi-view consistency in neural ray fields using parametric medial surfacesComputers and Graphics10.1016/j.cag.2024.103991123:COnline publication date: 21-Nov-2024
https://dl.acm.org/doi/10.1016/j.cag.2024.103991
Huang ZShi YGong M(2024)Visibility-Aware Pixelwise View Selection for Multi-View Stereo MatchingPattern Recognition10.1007/978-3-031-78456-9_9(130-144)Online publication date: 1-Dec-2024
https://dl.acm.org/doi/10.1007/978-3-031-78456-9_9
Peng RShen SXiong KGao HJiao JGu XWang R(2024)Surface-Centric Modeling for High-Fidelity Generalizable Neural Surface ReconstructionComputer Vision – ECCV 202410.1007/978-3-031-73411-3_11(183-200)Online publication date: 29-Sep-2024
https://dl.acm.org/doi/10.1007/978-3-031-73411-3_11
Younes MOuasfi ABoukhayma A(2024)SparseCraft: Few-Shot Neural Reconstruction Through Stereopsis Guided Geometric LinearizationComputer Vision – ECCV 202410.1007/978-3-031-72904-1_3(37-56)Online publication date: 29-Sep-2024
https://dl.acm.org/doi/10.1007/978-3-031-72904-1_3
Peng RGu XTang LShen SYu FWan ROh ANaumann TGloberson ASaenko KHardt MLevine S(2023)GenSProceedings of the 37th International Conference on Neural Information Processing Systems10.5555/3666122.3668611(56932-56945)Online publication date: 10-Dec-2023
https://dl.acm.org/doi/10.5555/3666122.3668611
Wei YLiu SZhou JLu J(2023)Depth-Guided Optimization of Neural Radiance Fields for Indoor Multi-View StereoIEEE Transactions on Pattern Analysis and Machine Intelligence10.1109/TPAMI.2023.326346445:9(10835-10849)Online publication date: 1-Sep-2023
https://dl.acm.org/doi/10.1109/TPAMI.2023.3263464
Show More Cited By

View Options

View options

Figures

Tables

Media

View Issue’s Table of Contents