research-article

Stereo Reconstruction from Multiperspective Panoramas

Authors:

Heung-Yeung Shum,

Chi-Keung Tang,

Richard SzeliskiAuthors Info & Claims

IEEE Transactions on Pattern Analysis and Machine Intelligence, Volume 26, Issue 1

Pages 45 - 62

https://doi.org/10.1109/TPAMI.2004.1261078

Published: 01 January 2004 Publication History

Abstract

Abstract--A new approach to computing a panoramic (360 degrees) depth map is presented in this paper. Our approach uses a large collection of images taken by a camera whose motion has been constrained to planar concentric circles. We resample regular perspective images to produce a set of multiperspective panoramas and then compute depth maps directly from these resampled panoramas. Our panoramas sample uniformly in three dimensions: rotation angle, inverse radial distance, and vertical elevation. The use of multiperspective panoramas eliminates the limited overlap present in the original input images and, thus, problems as in conventional multibaseline stereo can be avoided. Our approach differs from stereo matching of single-perspective panoramic images taken from different locations, where the epipolar constraints are sine curves. For our multiperspective panoramas, the epipolar geometry, to the first order approximation, consists of horizontal lines. Therefore, any traditional stereo algorithm can be applied to multiperspective panoramas with little modification. In this paper, we describe two reconstruction algorithms. The first is a cylinder sweep algorithm that uses a small number of resampled multiperspective panoramas to obtain dense 3D reconstruction. The second algorithm, in contrast, uses a large number of multiperspective panoramas and takes advantage of the approximate horizontal epipolar geometry inherent in multiperspective panoramas. It comprises a novel and efficient 1D multibaseline matching technique, followed by tensor voting to extract the depth surface. Experiments show that our algorithms are capable of producing comparable high quality depth maps which can be used for applications such as view interpolation.

References

[1]

S. Baker R. Szeliski and P. Anandan, “A Layered Approach to Stereo Reconstruction,” Proc. IEEE CS Conf. Computer Vision and Pattern Recognition, pp. 434-441, June 1998.

Digital Library

[2]

R. Benosman and J. Devars, “Panoramic Stereovision Vensor,” Panoramic Vision: Sensors, Theory, and Applications, pp. 161-168, New York: Springer, 2001.

Digital Library

[3]

S. Birchfield and C. Tomasi, “A Pixel Dissimilarity Measure that Is Insensitive to Image Sampling,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 20, no. 4, pp. 401-406, Apr. 1998.

Digital Library

[4]

R.C. Bolles H.H. Baker and D. Marimont, “Epipolar-Plane Image Analysis: An Approach to Determining Structure from Motion,” Int'l J. Computer Vision, vol. 1, pp. 7-55, 1987.

[5]

R.T. Collins, “A Space-Sweep Approach to True Multi-Image Matching,” Proc. IEEE CS Conf. Computer Vision and Pattern Recognition, pp. 358-363, June 1996.

Digital Library

[6]

S.J. Gortler R. Grzeszczuk R. Szeliski and M. F. Cohen, “The Lumigraph,” Proc. ACM SIGGRAPH '96, pp. 43-54, 1996.

Digital Library

[7]

R. Gupta and R. Hartley, “Linear Pushbroom Cameras,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 19, no. 9, pp. 963-975, Sept. 1997.

Digital Library

[8]

H. Ishiguro M. Yamamoto and S. Tsuji, “Omni-Directional Stereo,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 14,no. 2, pp. 257-262, Feb. 1992.

Digital Library

[9]

D. Jelinek and C.J. Taylor, “View Synthesis with Occlusion Reasoning Using Quasi-Sparse Feature Correspondences,” Proc. Seventh European Conf. Computer Vision, vol. 2, pp. 463-478, May 2002.

Digital Library

[10]

T. Kanade A. Yoshida K. Oda H. Kano and M. Tanaka, “A Stereo Machine for Video-Rate Dense Depth Mapping and Its New Applications,” Proc. IEEE CS Conf. Computer Vision and Pattern Recognition, pp. 196-202, June 1996.

Digital Library

[11]

S.B. Kang and R. Szeliski, “3-D Scene Data Recovery Using Omnidirectional Multibaseline Stereo,” Int'l J. Computer Vision, vol. 25, no. 2, pp. 167-183, Nov. 1997.

Digital Library

[12]

S.B. Kang R. Szeliski and J. Chai, “Handling Occlusions in Dense Multiview Stereo,” Proc. IEEE CS Conf. Computer Vision and Pattern Recognition, vol. 1, pp. 103-110, Dec. 2001.

[13]

R. Koch M. Pollefeys and L. Van Gool, “Multi Viewpoint Stereo from Uncalibrated Video Sequences,” Proc. Fifth European Conf. Computer Vision, vol. 1, pp. 55-71, June 1998.

Digital Library

[14]

V. Kolmogorov and R. Zabih, “Multi-Camera Scene Reconstruction via Graph Cuts,” Proc. Seventh European Conf. Computer Vision, vol. 3, pp. 82-96, May 2002.

Digital Library

[15]

K.N. Kutulakos and S.M. Seitz, “A Theory of Shape by Space Carving,” Proc. Seventh Int'l Conf. Computer Vision, pp. 307-314, Sept. 1999.

[16]

M. Levoy and P. Hanrahan, “Light Field Rendering,” Proc. ACM SIGGRAPH '96, pp. 31-42, 1996.

Digital Library

[17]

Y. Li C.-K. Tang and H.-Y. Shum, “Efficient Dense Depth Estimation from Dense Multiperspective Panoramas,” Proc. Eighth Int'l Conf. Computer Vision, pp. 119-126, 2001.

[18]

L. McMillan and G. Bishop, “Plenoptic Modeling: An Image-Based Rendering System,” Proc. ACM SIGGRAPH '95, pp. 39-46, 1995.

Digital Library

[19]

G. Medioni M. Lee and C. Tang, A Computational Framework for Feature Extraction and Segmentation. Amsterdam: Elsevier Science, 2000.

Digital Library

[20]

M. Okutomi and T. Kanade, “A Multiple Baseline Stereo,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 15, no. 4, pp. 353-363, Apr. 1993.

Digital Library

[21]

T. Pajdla, “Stereo with Oblique Cameras,” Int'l J. Computer Vision, vol. 47, nos. 1-3, pp. 161-170, Apr. 2002.

Digital Library

[22]

S. Peleg M. Ben-Ezra and Y. Pritch, “Omnistereo: Panoramic Stereo Imaging,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 23, no. 3, pp. 279-290, Mar. 2001.

Digital Library

[23]

S. Peleg and J. Herman, “Panoramic Mosaics by Manifold Projection,” Proc. IEEE CS Conf. Computer Vision and Pattern Recognition, pp. 338-343, June 1997.

Digital Library

[24]

P. Rademacher and G. Bishop, “Multiple-Center-of-Projection Images,” Proc. ACM SIGGRAPH '98, pp. 199-206, July 1998.

Digital Library

[25]

D. Scharstein and R. Szeliski, “Stereo Matching with Nonlinear Diffusion,” Int'l J. Computer Vision, vol. 28, no. 2, pp. 155-174, July 1998.

Digital Library

[26]

D. Scharstein and R. Szeliski, “A Taxonomy and Evaluation of Dense Two-Frame Stereo Correspondence Algorithms,” Int'l J. Computer Vision, vol. 47, no. 1, pp. 7-42, May 2002.

Digital Library

[27]

S. Seitz and C. Dyer, “Photorealistic Scene Reconstruction by Voxel Coloring,” Int'l J. Computer Vision, vol. 25, no. 3, Nov. 1999.

Digital Library

[28]

S. Seitz and J. Kim, “The Space of All Stereo Images,” Int'l J. Computer Vision, vol. 48, no. 1, pp. 21-38, June 2002.

Digital Library

[29]

H. Shum A. Kalai and S. Seitz, “Omnivergent Stereo,” Proc. Int'l Conf. Computer Vision, pp. 22-29, 1999.

[30]

H.-Y. Shum and L.-W. He, “Rendering with Concentric Mosaics,” Proc. ACM SIGGRAPH '99, pp. 299-306, 1999.

Digital Library

[31]

H.-Y. Shum and R. Szeliski, “Stereo Reconstruction from Multiperspective Panoramas,” Proc. Seventh Int'l Conf. Computer Vision, pp. 14-21, 1999.

[32]

R. Szeliski and P. Golland, “Stereo Matching with Transparency and Matting,” Int'l J. Computer Vision, special issue for Marr Prize papers, vol. 32, no. 1, pp. 45-61, Aug. 1999.

Digital Library

[33]

R. Szeliski and D. Scharstein, “Symmetric Sub-Pixel Stereo Matching,” Proc. Seventh European Conf. Computer Vision, vol. 2, pp. 525-540, May 2002.

Digital Library

[34]

D.N. Wood, et al. “Multiperspective Panoramas for Cel Animation,” Proc. ACM SIGGRAPH '97, pp. 243-250, Aug. 1997.

Digital Library

Cited By

Li MHu XDai JLi YDu S(2021)Omnidirectional stereo depth estimation based on spherical deep networkImage and Vision Computing10.1016/j.imavis.2021.104264114:COnline publication date: 1-Oct-2021
https://dl.acm.org/doi/10.1016/j.imavis.2021.104264
Oliveira MSappa ASantos V(2015)A Probabilistic Approach for Color Correction in Image Mosaicking ApplicationsIEEE Transactions on Image Processing10.1109/TIP.2014.237564224:2(508-523)Online publication date: 1-Feb-2015
https://dl.acm.org/doi/10.1109/TIP.2014.2375642
Yang HZhang HJin XMagnenat-Thalmann NYang HYang XBednarz TFujishiro I(2014)Modeling room structure from indoor panoramaProceedings of the 13th ACM SIGGRAPH International Conference on Virtual-Reality Continuum and its Applications in Industry10.1145/2670473.2670485(47-55)Online publication date: 30-Nov-2014
https://dl.acm.org/doi/10.1145/2670473.2670485
Show More Cited By

Index Terms

Stereo Reconstruction from Multiperspective Panoramas
1. Computing methodologies
  1. Artificial intelligence
    1. Computer vision
      1. Computer vision problems
        Reconstruction
  2. Computer graphics
    1. Image manipulation
2. Mathematics of computing
  1. Mathematical analysis
    1. Numerical analysis
      1. Computation of transforms

Recommendations

Virtual camera calibration and stereo correspondence of single-lens bi-prism stereovision system using geometrical approach

This paper proposes a simple geometrical ray approach to calibrate the extrinsic parameters of the virtual cameras and solve the stereo correspondence problem of the single-lens bi-prism stereovision system. Each images captured using this system can be ...
Omnivergent Stereo

The notion of a virtual camera for optimal 3D reconstruction is introduced. Instead of planar perspective images that collect many rays at a fixed viewpoint, omnivergent cameras collect a small number of rays at many different viewpoints. The resulting ...
Stereo Reconstruction from Polycentric Panoramas
RobVis '01: Proceedings of the International Workshop on Robot Vision

The paper defines polycentric panoramas as a generalized model of panoramic images which covers a wide range of previously introduced models such as single-center panoramas, multiperspective panoramic images, or concentric panoramic images. This paper ...

Comments

Information & Contributors

Information

Published In

cover image IEEE Transactions on Pattern Analysis and Machine Intelligence

IEEE Transactions on Pattern Analysis and Machine Intelligence Volume 26, Issue 1

January 2004

141 pages

ISSN:0162-8828

Issue’s Table of Contents

Copyright © Copyright © 2004 IEEE. All Rights Reserved.

Publisher

IEEE Computer Society

United States

Publication History

Published: 01 January 2004

Author Tags

Qualifiers

Research-article

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

14
Total Citations
View Citations
0
Total Downloads

Downloads (Last 12 months)0
Downloads (Last 6 weeks)0

Reflects downloads up to 25 Jan 2025

Other Metrics

View Author Metrics

Citations

Cited By

Li MHu XDai JLi YDu S(2021)Omnidirectional stereo depth estimation based on spherical deep networkImage and Vision Computing10.1016/j.imavis.2021.104264114:COnline publication date: 1-Oct-2021
https://dl.acm.org/doi/10.1016/j.imavis.2021.104264
Oliveira MSappa ASantos V(2015)A Probabilistic Approach for Color Correction in Image Mosaicking ApplicationsIEEE Transactions on Image Processing10.1109/TIP.2014.237564224:2(508-523)Online publication date: 1-Feb-2015
https://dl.acm.org/doi/10.1109/TIP.2014.2375642
Yang HZhang HJin XMagnenat-Thalmann NYang HYang XBednarz TFujishiro I(2014)Modeling room structure from indoor panoramaProceedings of the 13th ACM SIGGRAPH International Conference on Virtual-Reality Continuum and its Applications in Industry10.1145/2670473.2670485(47-55)Online publication date: 30-Nov-2014
https://dl.acm.org/doi/10.1145/2670473.2670485
Huang F(2014)Sensitivity analysis of pose recovery from multi-center panoramasMultimedia Tools and Applications10.1007/s11042-013-1438-472:2(1193-1213)Online publication date: 1-Sep-2014
https://dl.acm.org/doi/10.1007/s11042-013-1438-4
Sturm PRamalingam STardif JGasparini SBarreto J(2011)Camera Models and Fundamental Concepts Used in Geometric Computer VisionFoundations and Trends® in Computer Graphics and Vision10.1561/06000000236:1–2(1-183)Online publication date: 1-Jan-2011
https://dl.acm.org/doi/10.1561/0600000023
Zhang CJia JLiu ZJorge JPan ZZhang XAu ODong W(2011)Image based stereoscopic rendering with non-mosaic impeller matchingProceedings of the 10th International Conference on Virtual Reality Continuum and Its Applications in Industry10.1145/2087756.2087859(541-544)Online publication date: 11-Dec-2011
https://dl.acm.org/doi/10.1145/2087756.2087859
Trinh HKim DKang SJo K(2009)Window extraction using geometrical characteristics of building surfaceProceedings of the 5th international conference on Emerging intelligent computing technology and applications10.5555/1802674.1802750(585-594)Online publication date: 16-Sep-2009
https://dl.acm.org/doi/10.5555/1802674.1802750
Zheng JShi M(2008)Scanning Depth of Route Panorama Based on Stationary BlurInternational Journal of Computer Vision10.1007/s11263-007-0080-x78:2-3(169-186)Online publication date: 1-Jul-2008
https://dl.acm.org/doi/10.1007/s11263-007-0080-x
Jiali ZYongdong ZYanfei SGuangnan N(2007)Panoramic video coding using affine motion compensated predictionProceedings of the 2007 international conference on Multimedia content analysis and mining10.5555/1776594.1776614(112-121)Online publication date: 30-Jun-2007
https://dl.acm.org/doi/10.5555/1776594.1776614
Zheng JShi M(2007)Depth from stationary blur with adaptive filteringProceedings of the 8th Asian conference on Computer vision - Volume Part II10.5555/1775728.1775735(42-52)Online publication date: 18-Nov-2007
https://dl.acm.org/doi/10.5555/1775728.1775735
Show More Cited By

View Options

View options

Figures

Tables

Media

View Issue’s Table of Contents